[Dune-devel] [Dune] DUNE migration to Git complete

[Markus Blatt]

On Tue, May 07, 2013 at 01:01:03PM +0200, Steffen Müthing wrote:
>
> I just talked about this with Christian. My reasons for adding it were
> 
> - Having a free publicly accessible backup in case there are ever any problems with the server in Heidelberg.
>   In theory, every developer's repository is a backup already, but depending on how he / she synchronizes with
>   the server, there will often be missing / outdated branches (that happens to remote branches without a local
>   tracking branch if you only ever pull and never fetch from the remote). The GitHub backup will always be up-to-date
>   with the official servers.

Having a backup is a good thing. But having it on such a site is not
for the reasons I already quoted.

> - Reserving the name on GitHub, just in case. Admittedly, this would not have required us to actually mirror the
>   data, but setting it up took about an hour… and it adds visibility: GitHub has become a kind of standard
>   goto place for open source software.

There have been a lot of these in recent years: freshmeat,
sourceforge, you name it. 

> - Something that Markus mentioned a while back (I can't remember whether it was on the list or on FlySpray):
>   People seem to really grok the way GitHub does collaboration through their pull request system. I was skeptical
>   about it at first, but I tried it and it's really slick and works very well.
> 

I do not believe that this is only possible with github. To me this
seems merely an automated way of creating a diff and seeing whether
it merges. Andreas Lauser has provided quite a few git-diffs through
flyspray in the last year. I am sure there is a tool for displaying such
diffs more readable. I saw at least one when I investigated buildbot
that allowed even for comments. (but it was Java :/). I will try to
recall the name (reading further down I am sure it was gerrit).

> As to why GitHub:
> [...]
> 
> 
> I discussed the issue of using GitHub for pull requests with Christian over the phone and it is kind of a double-edged sword:
> 
> On the one hand, when / if people start to use GitHub pull request to communicate their changes, we add another
> workflow that is totally separate from our existing setup with FlySpray and patch attachments with all the associated
> risks: Additional work, patches falling through the cracks, fragmentation of information between FlySpray and the
> pull requests on GitHub…
> 
> On the other hand, while our FlySpray works pretty well for people submitting patches (they are linked right in
> the comments, you can click on them and directly see what they do), pull requests are a different story. Take a look
> at FS#1287 for example: I added a pull request to that bug, which Christian promptly merged. But this was a small,
> trivial patch queue: One patch, and that one was a one liner. Imagine a more complicated pull request with something
> like 10 commits. Now you need to find the associated branch (maybe looking it up in the repository viewer) to be
> able to see the changes. Let's say you don't like something about patch 5/10 and you write a comment in FlySpray:
> How do you reference the commit? The only sane way is by its sha1. Now I read your comment, but in order to know
> what you are talking about, I have to somehow look up the commit again, which takes time and breaks concentration.
> We could probably work around some of these problems by posting links to the repository viewer instead of simple
> branch names / commit ids, but that would add a lot of clutter to the comments. And things become even more complicated
> if an external contributor requests a merge from his/her own repository…
> The whole thing is just not funny when you compare that workflow with the really slick experience offered by GitHub's
> pull requests. ;-)
> 
> 

OPM uses github. Some very disciplined developers really use the comment
functions on code lines, but that is not done very often. You are
talking about complicated multi-commit patches. Most people will
rather glance at these patches than reading them properly and not
comment on specific lines. The more important aspect is that even
those reading it cannot oversee all the side effects. One has to apply
them, compile and test. (BTW: this is possible with a try scheduler of buildbot)

> 
> I think what I'm trying to say is this: We'll have to think about how we handle pull requests, as people will start sending
> those (it's just more convenient than patch files). We have a number of options there:
> 
> [ snipped valid points about bug trackers]

There are quite a few things that really bug me about the github bug
tracker, most importantly one cannot conveniently attach anything to
it. Which is mandatory for posting bugs, I guess.


> - Find a new tool that we can install locally to handle pull requests. Unfortunately, the only one that comes to my mind is Gerrit, but
>   Gerrit would definitely be absolute overkill for our purposes - I wouldn't want to set up or administer that beast. But if someone knows
>   about a different package that might fit our needs, just say the word! Depending on whether we can set up some kind of automatic data
>   transfer, this would also end up introducing two different patch workflows.
> 

That is why I never dared to try Gerrit.

> - Use the pull request support offered by one of the big code hosting sites (GitHub, Gitorious, BitBucket). Again, some thoughts on that:
>   - They tend to have really nice interfaces (especially GitHub and BitBucket).
>   - We introduce a completely external component in our setup, over which we don't really have any control. This carries the risk of their site
>     changing in ways we don't like, not having the data available
>   - locally, their site simply disappearing etc.

Of course we need some kind of backup, but this should be manageable.

>   - There is a good chance contributors are already comfortable with the way the process works on these public sites, simply because of
>     their ubiquity.

True point

>   - Because Christian already asked: We cannot just move our repositories completely to one of those sites if we want to retain things like
>     per-branch commit rights and the whitespace check.

I do not think that per-branch commit rights are necessary. People
just fork, create a branch on the forked repository.

Concerning the whitespace check:

It breaks most of the things that you want to achieve with a
mirror. Just imagine you get a pull request with trailing
whitespace... How can you resolve this without offending the
contributor?


>   - We would have two workflows that live side-by-side, making it harder to look for stuff. Christian had the idea of mitigating this by using
>     GitHub's notifications to automatically mirror the pull requests
>     into our FlySpray in some way.

Guys, it gets more and more complicated, more and more fragile and
more and more unmanageable.

>   - People who want to keep using the FlySpray the way they have always done can continue to do so.
>   - At least some of the developers would have to be willing to look at those pull requests (there is no need to actively look for them, GitHub
>     automatically sends out email notifications when told to do so).
> 
> I'm really not sure which of the options to pick (perhaps someone has a better idea?), but I think I would like to try out the GitHub route
> because
> a) It doesn't require a lot of setup work
> b) After playing with it for a little while, their interface *seems* to work really well, and I think it would save developer's who invest the time to
>    look at user patches some of that precious time they invested (and we all know that there is never enough of that).
> 

To sum up my opinion:

I am not against using Github or anything else per se, but I heavily
dislike Githubs bug tracker. Additionally, if we choose such a system it
should be the primary entry point relieving us from administrating our
own repositories and not putting more burden on us.

Just my 2cents,

Markus

-- 
Do you need more support with DUNE or HPC in general? 

Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de
Hans-Bunte-Str. 8-10, 69123 Heidelberg, Germany
Tel.: +49 (0) 160 97590858  Fax: +49 (0)322 1108991658