[Dune-devel] [Dune] DUNE migration to Git complete

Steffen Müthing steffen.muething at ipvs.uni-stuttgart.de
Tue May 7 13:01:03 CEST 2013 

Hello everybody,

I just realized that this mail got a little long… please bear with me!

Am 07.05.2013 um 09:54 schrieb Oliver Sander:

> 
>>> - In order to mirror our repositories on GitHub, I set up an organizational account called dune-project.
>>>   Currently I am the sole owner of the organization, but I'll gladly add other developers. Just let me know
>>>   and tell me your GitHub account name. I'll also add you to the alias for the associated email account
>>>   (github at dune-project.org).
>> 
>> Why would we need a github mirror? And what about gitorious, launchpad
>> etc.? Do we really want to do this?
> 
> Same question here.  What do we need the github mirror for?
> Best,
> Oliver

I just talked about this with Christian. My reasons for adding it were

- Having a free publicly accessible backup in case there are ever any problems with the server in Heidelberg.
  In theory, every developer's repository is a backup already, but depending on how he / she synchronizes with
  the server, there will often be missing / outdated branches (that happens to remote branches without a local
  tracking branch if you only ever pull and never fetch from the remote). The GitHub backup will always be up-to-date
  with the official servers.

- Reserving the name on GitHub, just in case. Admittedly, this would not have required us to actually mirror the
  data, but setting it up took about an hour… and it adds visibility: GitHub has become a kind of standard
  goto place for open source software.

- Something that Markus mentioned a while back (I can't remember whether it was on the list or on FlySpray):
  People seem to really grok the way GitHub does collaboration through their pull request system. I was skeptical
  about it at first, but I tried it and it's really slick and works very well.

As to why GitHub:
- It seems to be the most popular one
- Gitorious feels *very* clunky to me in comparison (and also lacks features)
- BitBucket is run by Atlassian, which shows up in arbitrarily limited features to make people buy their professional tools. On the other hand,
  as an open source project, we could get free access to those tools (either hosted or on our own server). BitBucket also offers unlimited private
  repositories to academic users (great for things like paper repositories). But compared to GitHub, their interface often feels sllooooww to me ;-)
- Launchpad only works with bazaar. And I don't like the whole system - I had some experience with it through FEniCS, and it always seemed
  really weird.


I discussed the issue of using GitHub for pull requests with Christian over the phone and it is kind of a double-edged sword:

On the one hand, when / if people start to use GitHub pull request to communicate their changes, we add another
workflow that is totally separate from our existing setup with FlySpray and patch attachments with all the associated
risks: Additional work, patches falling through the cracks, fragmentation of information between FlySpray and the
pull requests on GitHub…

On the other hand, while our FlySpray works pretty well for people submitting patches (they are linked right in
the comments, you can click on them and directly see what they do), pull requests are a different story. Take a look
at FS#1287 for example: I added a pull request to that bug, which Christian promptly merged. But this was a small,
trivial patch queue: One patch, and that one was a one liner. Imagine a more complicated pull request with something
like 10 commits. Now you need to find the associated branch (maybe looking it up in the repository viewer) to be
able to see the changes. Let's say you don't like something about patch 5/10 and you write a comment in FlySpray:
How do you reference the commit? The only sane way is by its sha1. Now I read your comment, but in order to know
what you are talking about, I have to somehow look up the commit again, which takes time and breaks concentration.
We could probably work around some of these problems by posting links to the repository viewer instead of simple
branch names / commit ids, but that would add a lot of clutter to the comments. And things become even more complicated
if an external contributor requests a merge from his/her own repository…
The whole thing is just not funny when you compare that workflow with the really slick experience offered by GitHub's
pull requests. ;-)



I think what I'm trying to say is this: We'll have to think about how we handle pull requests, as people will start sending
those (it's just more convenient than patch files). We have a number of options there:

- Try to find a way to make them work in FlySpray. For the time being, this would boil down to a set of best practices to
  be followed by people when posting pull requests or discussing them, things like: Post links to a repository viewer for
  the branch, let people know when you reworked your patches (possibly with updated commit links) etc. The FlySpray bugtracker
  lists Git support as a planned feature for version 2.0, but there is no ETA on that (they are still working on releasing 1.0).

- Switch to a bug tracker with Git integration (e.g. the one in Redmine, or JIRA or whatever) which recognizes branch names
  and commit ids and turns them into links. See the PDELab bug tracker for example (even though there is not a lot of useful
  examples on there, but I could add some). Some thoughts on that:
  - We would need to somehow migrate old issues. There are different ways to do that, with different levels of convenience and
    effort involved in setting them up.
  - Redmine supports inter-project links, so external people could link to their clone of the module to reference their commits,
    but only if they have a clone of the repository in the same Redmine instance. I don't know whether Münster wants to support
    this (I have my doubts… ;-) ).
  - People are used to the FlySpray bugtracker, switching would require learning the new one - and the Redmine bug tracker has
    its quirks as well.
  - All information would still be in one place, albeit a different one.
  - Other issues would benefit from the ability to reference commits in bug comments as well (imagine opening an issue saying that
    commit a8782b broke Mac support - with Redmine you would be able to click that commit id and immediately see the broken commit).

- Find a new tool that we can install locally to handle pull requests. Unfortunately, the only one that comes to my mind is Gerrit, but
  Gerrit would definitely be absolute overkill for our purposes - I wouldn't want to set up or administer that beast. But if someone knows
  about a different package that might fit our needs, just say the word! Depending on whether we can set up some kind of automatic data
  transfer, this would also end up introducing two different patch workflows.

- Use the pull request support offered by one of the big code hosting sites (GitHub, Gitorious, BitBucket). Again, some thoughts on that:
  - They tend to have really nice interfaces (especially GitHub and BitBucket).
  - We introduce a completely external component in our setup, over which we don't really have any control. This carries the risk of their site
    changing in ways we don't like, not having the data available locally, their site simply disappearing etc.
  - There is a good chance contributors are already comfortable with the way the process works on these public sites, simply because of
    their ubiquity.
  - Because Christian already asked: We cannot just move our repositories completely to one of those sites if we want to retain things like
    per-branch commit rights and the whitespace check.
  - We would have two workflows that live side-by-side, making it harder to look for stuff. Christian had the idea of mitigating this by using
    GitHub's notifications to automatically mirror the pull requests into our FlySpray in some way.
  - People who want to keep using the FlySpray the way they have always done can continue to do so.
  - At least some of the developers would have to be willing to look at those pull requests (there is no need to actively look for them, GitHub
    automatically sends out email notifications when told to do so).

I'm really not sure which of the options to pick (perhaps someone has a better idea?), but I think I would like to try out the GitHub route
because
a) It doesn't require a lot of setup work
b) After playing with it for a little while, their interface *seems* to work really well, and I think it would save developer's who invest the time to
   look at user patches some of that precious time they invested (and we all know that there is never enough of that).


Sorry for the long mail, but this is something we should think about - otherwise we will probably run into workload
issues (which will likely be resolved by not merging people's patches, which IMHO is bad).

Greetings,

Steffen



> 
>> 
>>> - We should really encourage users to send us patches via pull request (preferred) or, if that is not possible
>>>   for some reason, via "git format-patch". Apart from saving the person applying the patch a lot of work, this
>>>   also has the additional advantage of preserving the committer (for pull requests) or at least author (for
>>>   "git format-patch") information. With that information in the repository it should become a lot easier to compile
>>>   stuff like lists of external contributors in the future.
>> 
>> agree
>> 
>> Ciao
>> Christian
>> 
>> _______________________________________________
>> Dune-devel mailing list
>> Dune-devel at dune-project.org
>> http://lists.dune-project.org/mailman/listinfo/dune-devel
> 
> 
> _______________________________________________
> Dune-devel mailing list
> Dune-devel at dune-project.org
> http://lists.dune-project.org/mailman/listinfo/dune-devel

Steffen Müthing
Universität Stuttgart
Institut für Parallele und Verteilte Systeme
Universitätsstr. 38
70569 Stuttgart
Tel: +49 711 685 88429
Fax: +49 711 685 88340
Email: steffen.muething at ipvs.uni-stuttgart.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://lists.dune-project.org/pipermail/dune-devel/attachments/20130507/1b10483c/attachment.sig>

More information about the Dune-devel mailing list