[Dune-devel] Gitlab: Migrate artifacts, packages, (dependency proxy) to S3-compat object storage

[Oliver Sander]

Thanks Ansgar!

I do wonder whether the >150GB of artifacts are still needed? Is it 
feasible to do some sort of garbage collection here?

Best,
Oliver

On 6/15/26 17:45, Ansgar Burchardt wrote:
> Hi,
> 
> the migration is done. I also had to allow the shared CI runner to 
> access the S3 domain.
> 
> Disk usage now looks like this:
> 
> # df -h /srv/data
> 
> Filesystem      Size  Used Avail Use% Mounted on
> 
> /dev/sdb1       300G   63G  231G  22% /srv/data
> 
> 
> And on the S3 side:
> 
> | Bucket name                  | Number of objects | Size        |
> 
> |------------------------------|-------------------|-------------|
> 
> | dune-gitlab-artifacts        |            667385 | 171.222 GiB |
> 
> | dune-gitlab-dependency-proxy |                 0 |   0.000 GiB |
> 
> | dune-gitlab-packages         |              7970 |  44.667 GiB |
> 
> | dune-gitlab-registry         |            118906 | 170.384 GiB |
> 
> |------------------------------|-------------------|-------------|
> 
> | ∑                            |            794259 | 386.263 GiB |
> 
> 
> We currently have a quota of 2.5 TB.
> 
> I also configured a policy that objects in S3 will only be marked for 
> deletion and only be removed after 90 days. This should be sufficient in 
> case Gitlab deletes data due to a software malfunction.
> 
> 
> Some inconsistencies were found (and still exist):
> 
> ~7 GiB of CI artifacts on disk, but missing in database (not moved to S3).
> 
> ~40 packages in database, but missing from disk.
> 
> Ansgar
> 
> On Thu, 2026-06-11 at 15:34 +0000, Ansgar Burchardt wrote:
>> Hi,
>>
>> the virtual disk for Gitlab data is fairly full:
>>
>> root at dune-gitlab <mailto:root at dune-gitlab>:~# df -h /srv/data
>> Filesystem      Size  Used Avail Use% Mounted on
>> /dev/sdb1       300G  264G   32G  90% /srv/data
>>
>> A large chunk is CI job artifacts and packages:
>>
>> root at dune-gitlab <mailto:root at dune-gitlab>:~# du -shc /srv/data/gitlab/gitlab-rails/shared/*
>> 157G	/srv/data/gitlab/gitlab-rails/shared/artifacts
>> 3.3G	/srv/data/gitlab/gitlab-rails/shared/cache
>> 0	/srv/data/gitlab/gitlab-rails/shared/ci_secure_files
>> 0	/srv/data/gitlab/gitlab-rails/shared/dependency_proxy
>> 0	/srv/data/gitlab/gitlab-rails/shared/encrypted_settings
>> 0	/srv/data/gitlab/gitlab-rails/shared/external-diffs
>> 2.7G	/srv/data/gitlab/gitlab-rails/shared/lfs-objects
>> 47G	/srv/data/gitlab/gitlab-rails/shared/packages
>> 0	/srv/data/gitlab/gitlab-rails/shared/pages
>> 0	/srv/data/gitlab/gitlab-rails/shared/registry
>> 0	/srv/data/gitlab/gitlab-rails/shared/terraform_state
>> 0	/srv/data/gitlab/gitlab-rails/shared/tmp
>> 209G	total
>>
>> For comparison the Git repository data and Gitlab database are much 
>> smaller:
>>
>> root at dune-gitlab <mailto:root at dune-gitlab>:~# du -shc /srv/data/gitlab/git-data /srv/data/gitlab/postgresql
>> 14G	/srv/data/gitlab/git-data
>> 15G	/srv/data/gitlab/postgresql
>> 29G	total
>>
>> Instead of resizing the disk, I would like to migrate some more data 
>> to S3-compatible object storage. We already use object storage for the 
>> Docker registry.
>>
>> I would migrate the following objects:
>>
>> 1. Job artifacts (CI/CD job artifacts including logs).
>> 2. Project packages (PyPI, ...)
>> 3. Dependency proxy (currently unused, just configure it to use object 
>> storage).
>>
>> I would keep Git LFS objects on the disk as the S3-compat storage has 
>> redundancy (to cover disk failure), but no full backups (when 
>> something deletes too much). I don't think losing CI/CD job artifacts 
>> or packages is too bad, just like with Docker images.
>>
>> The migration should be possible while Gitlab runs. It is also 
>> possible to migrate back to local storage.
>>
>> See the Gitlab documentation for details: https://docs.gitlab.com/ 
>> administration/object_storage/ <https://docs.gitlab.com/ 
>> administration/object_storage/>
>>
>> Regards,
>> Ansgar
>>
> 
> _______________________________________________
> Dune-devel mailing list
> Dune-devel at lists.dune-project.org
> https://lists.dune-project.org/mailman/listinfo/dune-devel