[Dune-devel] [Dune] DUNE migration to Git complete

[Steffen Müthing]

Am 07.05.2013 um 14:25 schrieb Markus Blatt:

> On Tue, May 07, 2013 at 01:01:03PM +0200, Steffen Müthing wrote:
>> 
>> I just talked about this with Christian. My reasons for adding it were
>> 
>> - Having a free publicly accessible backup in case there are ever any problems with the server in Heidelberg.
>>  In theory, every developer's repository is a backup already, but depending on how he / she synchronizes with
>>  the server, there will often be missing / outdated branches (that happens to remote branches without a local
>>  tracking branch if you only ever pull and never fetch from the remote). The GitHub backup will always be up-to-date
>>  with the official servers.
> 
> Having a backup is a good thing. But having it on such a site is not
> for the reasons I already quoted.

It's unfortunate that we can't put some kind of longer text on the repository pages there apart from the contents of the README
file, I agree. I just tweaked the repositories to at least display our website and added a note in all caps to visit dune-project.org.

> 
>> - Something that Markus mentioned a while back (I can't remember whether it was on the list or on FlySpray):
>>  People seem to really grok the way GitHub does collaboration through their pull request system. I was skeptical
>>  about it at first, but I tried it and it's really slick and works very well.
>> 
> 
> I do not believe that this is only possible with github. To me this
> seems merely an automated way of creating a diff and seeing whether
> it merges. Andreas Lauser has provided quite a few git-diffs through
> flyspray in the last year. I am sure there is a tool for displaying such
> diffs more readable. I saw at least one when I investigated buildbot
> that allowed even for comments. (but it was Java :/). I will try to
> recall the name (reading further down I am sure it was gerrit).

As I said, as long as people use patch files, FlySpray works pretty well. The problems arise when people start submitting
pull requests.

> 
> OPM uses github. Some very disciplined developers really use the comment
> functions on code lines, but that is not done very often. You are
> talking about complicated multi-commit patches. Most people will
> rather glance at these patches than reading them properly and not
> comment on specific lines. The more important aspect is that even
> those reading it cannot oversee all the side effects. One has to apply
> them, compile and test. (BTW: this is possible with a try scheduler of buildbot)

The thing that I really liked when trying it was how easy it was to inspect the diffs and to have them right there with
the comments. I just felt that I made things a lot easier - even if you only comment on the pull request itself as a whole or
at the commit-level. Of course you still have to download a more complicated pull request and test it, but that is also
a lot easier and quicker with a pull request than with a collection of patches, especially as those get updated to
incorporate the input from your comments.

> 
>> 
>> I think what I'm trying to say is this: We'll have to think about how we handle pull requests, as people will start sending
>> those (it's just more convenient than patch files). We have a number of options there:
>> 
>> [ snipped valid points about bug trackers]
> 
> There are quite a few things that really bug me about the github bug
> tracker, most importantly one cannot conveniently attach anything to
> it. Which is mandatory for posting bugs, I guess.

I thought you could just drag a file on the comment field to attach it? But yes, their bug tracker is very limited.

> 
> 
>> - Find a new tool that we can install locally to handle pull requests. Unfortunately, the only one that comes to my mind is Gerrit, but
>>  Gerrit would definitely be absolute overkill for our purposes - I wouldn't want to set up or administer that beast. But if someone knows
>>  about a different package that might fit our needs, just say the word! Depending on whether we can set up some kind of automatic data
>>  transfer, this would also end up introducing two different patch workflows.
>> 
> 
> That is why I never dared to try Gerrit.

Also, Gerrit's raison d'être is to enforce rigorous code review, with people having to sign off on other people's changes, requiring integration
builds to pass before merging and stuff like that - you would hate it! ;-)

> 
>>  - Because Christian already asked: We cannot just move our repositories completely to one of those sites if we want to retain things like
>>    per-branch commit rights and the whitespace check.
> 
> I do not think that per-branch commit rights are necessary. People
> just fork, create a branch on the forked repository.

This is politics again, but in the SVN repos, there were things like only allowing release managers to commit to release branches. On GitHub,
write access is a binary switch, it's either off or on. Moreover, GitHub also lets you happily do things like non fast-forward pushes, which are
*really* dangerous if you don't exactly know what you are doing. So having a "push gateway" is also some kind of security net.

And as quite a number of developers seem to like the idea of keeping trailing whitespace out of the repositories via an update hook, using GitHub
directly just isn't possible (in fact, I don't think any hosted service will let you run hooks on their servers, for good reasons).

> 
> Concerning the whitespace check:
> 
> It breaks most of the things that you want to achieve with a
> mirror. Just imagine you get a pull request with trailing
> whitespace... How can you resolve this without offending the
> contributor?

I thought we mostly resolved that with our local commit hook. If someone sends me a pull request, I would really hope the contributor at least
tried to test whether the module compiles. For that, he/she needs to run dunecontrol. As long as the contributor do the normal thing and runs
dunecontrol right after cloning, the local whitespace hook will make sure that the locally created commits also pass the server-side hook (the
problems you encountered with merges in PDELab cannot occur in the core modules because the history was cleaned up).

To circumvent that protection, the contributor has to either deactivate the hook on purpose (in that case, they have no right to complain) or
to do things in the following order, which I would consider possible, but unlikely:

clone -> write patch -> create commits -> run dunecontrol

I also added reminders to always start by running dunecontrol before creating patches to several places on the website.

> 
> 
>>  - We would have two workflows that live side-by-side, making it harder to look for stuff. Christian had the idea of mitigating this by using
>>    GitHub's notifications to automatically mirror the pull requests
>>    into our FlySpray in some way.
> 
> Guys, it gets more and more complicated, more and more fragile and
> more and more unmanageable.

That was just an idea to avoid data fragmentation, it's by no means a must…
A more lightweight way of making sure you get to see all patch discussions without having to sign up to GitHub would be to
simply resend GitHub's notification emails on the dune-bugs list (and yes I know, that will be controversial too ;-) ).

> 
>>  - People who want to keep using the FlySpray the way they have always done can continue to do so.
>>  - At least some of the developers would have to be willing to look at those pull requests (there is no need to actively look for them, GitHub
>>    automatically sends out email notifications when told to do so).
>> 
>> I'm really not sure which of the options to pick (perhaps someone has a better idea?), but I think I would like to try out the GitHub route
>> because
>> a) It doesn't require a lot of setup work
>> b) After playing with it for a little while, their interface *seems* to work really well, and I think it would save developer's who invest the time to
>>   look at user patches some of that precious time they invested (and we all know that there is never enough of that).
>> 
> 
> To sum up my opinion:
> 
> I am not against using Github or anything else per se, but I heavily
> dislike Githubs bug tracker. Additionally, if we choose such a system it
> should be the primary entry point relieving us from administrating our
> own repositories and not putting more burden on us.

Well, I share your dislike of GitHub's bugtracker ;-). Between that one and FlySpray, I infinitely prefer the latter. But as I have a dissertation
to finish, replacing a bug tracker (or even researching a FlySpray replacement) isn't something I have time to do in the near future.

I would suggest to simply try the GitHub workflow with some pull requests (I volunteer to create those, I have some patches for the core modules
lying around). Then we'd just need a developer (or perhaps even several) who are willing to sign up to GitHub and add the Dune repos to their
"watched" list.

It's just like the private branches - I don't think arguing about theoretical merits and problems gets us very far, we simply have to see what works
- of course, that implies being willing to go back if it doesnt. And since the setup cost is close to zero, there isn't much to lose if we just do a kind
of closed beta tryout, is there? Well, for GitHub it's not really a closed beta, but we'll just assume no external people will jump on the bandwagon
right away… ;-)

Steffen

> 
> Just my 2cents,
> 
> Markus
> 
> -- 
> Do you need more support with DUNE or HPC in general? 
> 
> Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de
> Hans-Bunte-Str. 8-10, 69123 Heidelberg, Germany
> Tel.: +49 (0) 160 97590858  Fax: +49 (0)322 1108991658 
> 
> _______________________________________________
> Dune-devel mailing list
> Dune-devel at dune-project.org
> http://lists.dune-project.org/mailman/listinfo/dune-devel

Steffen Müthing
Universität Stuttgart
Institut für Parallele und Verteilte Systeme
Universitätsstr. 38
70569 Stuttgart
Tel: +49 711 685 88429
Fax: +49 711 685 88340
Email: steffen.muething at ipvs.uni-stuttgart.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://lists.dune-project.org/pipermail/dune-devel/attachments/20130507/ef4a6f1f/attachment.sig>