<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Steffen,<br>
<br>
thank you for the instructions and scripts!<br>
<br>
Kind regards<br>
Bernd<br>
<br>
On 11/02/2015 03:44 PM, Steffen Müthing wrote:<br>
</div>
<blockquote
cite="mid:34D5587E-2D8C-46E9-A885-054A888AF427@iwr.uni-heidelberg.de"
type="cite">
<pre wrap="">Hi Bernd,
</pre>
<blockquote type="cite">
<pre wrap="">Am 30.10.2015 um 08:09 schrieb Bernd Flemisch <a class="moz-txt-link-rfc2396E" href="mailto:bernd@iws.uni-stuttgart.de"><bernd@iws.uni-stuttgart.de></a>:
Hi Steffen,
---- Steffen Müthing schrieb ----
</pre>
<blockquote type="cite">
<pre wrap="">I have also taken a look at the Flyspray bugs - I think we’ll be able to move them over to
</pre>
</blockquote>
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">GitLab without major problems.
</pre>
</blockquote>
<pre wrap="">Since we recently also moved to GitLab, it would be very interesting if you could share some details on how to move the Flyspray tasks.
</pre>
</blockquote>
<pre wrap="">
moving the issues wasn’t as bad as I feared. Here is a quick outline:
- You need access to the PostgreSQL databases of both Flyspray and GitLab and the Flyspray filesystem with the stored
attachments.
- There shouldn’t be any issues in the target project before starting the conversion. That way, the bug numbers in Flyspray and GitLab will
match. That’s why we created the flyspray group with the FS repository (and because flyspray/FS#1234 are nicely readable links in GitLab).
Step 1: Upload the attachments
Attachments in GitLab are always associated with a project and are stored with some hash-like URL. Unfortunately, GitLab doesn’t have API calls
for creating attachments, so this is a little fiddly, you have to reuse the session from your browser to do the uploads.
So I did the following:
- Enable the wiki on the FS project
- Go to the wiki section. You can directly edit the default wiki page, and below the text field, there is an upload button.
- Before doing anything else, turn on the developer tools and network request logging in your browser.
- Reload the page
- Upload a small file with the upload button. As a result, you will see a link pasted into the text field.
- In the network logs of the developer tools, you should see an XHR request to a URL ending in uploads. That has two important headers, one for
the session cookie and some X-CSRF-Token. You need the URL, the session cookie and the token
- I’ve pasted all three of those into a Python script that also knows how to access the Flyspray database and the directory with the uploaded Flyspray
attachments.
- The script will then upload all attachments and dump some information about the uploaded files into a state file that will be used in step 2.
Step 2: Create tasks and comments
- This can be done through the GitLab API
- As an administrator, look up the private token in your GitLab profile
- You need to create a map from Flyspray email addresses to GitLab user names
- You will also need a GitLab user that acts as author for users that are in Flyspray, but not in GitLab
- Optional: maps from things like Flyspray task types, operating systems and resolution types to GitLab issue labels. For this, you’ll have to poke around
in the Flyspray SQL tables.
I’ve written another Python script that creates all of the issues and comments through the GitLab API. For the ~1700 issues and 9.000 comments from the
DUNE Flyspray instance, this took about an hour.
The script again dumps some metadata for step 3.
Step 3: Fix dates:
GitLab doesn’t let you forge creation dates through the API, so we have to fiddle with SQL to work around that. My third Python script thus creates lots
and lots of SQL UPDATE statements that have to be run against the GitLab database to fix up the creation, closing, and update dates.
I’ve attached the three scripts for reference. The scripts all require Python 3.4 and the python packages requests, arrow and postgres (best installed in a virtualenv
through pip). Looking at the scripts, you’ll also have to adjust some GitLab project ids, GitLab URLs and paths to state files with metadata. The scripts aren’t pretty,
but they did the job. ;-)
Best,
Steffen
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">
Thank you. Kind regards
Bernd
_______________________________________________
</pre>
<blockquote type="cite">
<pre wrap="">Dune-devel mailing list
</pre>
</blockquote>
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap=""><a class="moz-txt-link-abbreviated" href="mailto:Dune-devel@dune-project.org">Dune-devel@dune-project.org</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap=""><a class="moz-txt-link-freetext" href="http://lists.dune-project.org/mailman/listinfo/dune-devel">http://lists.dune-project.org/mailman/listinfo/dune-devel</a>
</pre>
</blockquote>
<pre wrap="">
_______________________________________________
Dune-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Dune-devel@dune-project.org">Dune-devel@dune-project.org</a>
<a class="moz-txt-link-freetext" href="http://lists.dune-project.org/mailman/listinfo/dune-devel">http://lists.dune-project.org/mailman/listinfo/dune-devel</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
_______________________________________________________________
Bernd Flemisch phone: +49 711 685 69162
IWS, Universität Stuttgart fax: +49 711 685 60430
Pfaffenwaldring 61 email: <a class="moz-txt-link-abbreviated" href="mailto:bernd@iws.uni-stuttgart.de">bernd@iws.uni-stuttgart.de</a>
D-70569 Stuttgart url: <a class="moz-txt-link-abbreviated" href="http://www.hydrosys.uni-stuttgart.de">www.hydrosys.uni-stuttgart.de</a>
_______________________________________________________________
</pre>
</body>
</html>