<html><head><style>pre,code,address {
margin: 0px;
}
h1,h2,h3,h4,h5,h6 {
margin-top: 0.2em;
margin-bottom: 0.2em;
}
ol,ul {
margin-top: 0em;
margin-bottom: 0em;
}
blockquote {
margin-top: 0em;
margin-bottom: 0em;
}
</style><style>pre,code,address {
margin: 0px;
}
h1,h2,h3,h4,h5,h6 {
margin-top: 0.2em;
margin-bottom: 0.2em;
}
ol,ul {
margin-top: 0em;
margin-bottom: 0em;
}
blockquote {
margin-top: 0em;
margin-bottom: 0em;
}
</style></head><body><div>Hi,</div><div><br></div><div>the migration is done. I also had to allow the shared CI runner to access the S3 domain.</div><div><br></div><div>Disk usage now looks like this:</div><div><br></div><pre># df -h /srv/data</pre><pre>Filesystem Size Used Avail Use% Mounted on</pre><pre>/dev/sdb1 300G 63G 231G 22% /srv/data</pre><pre><br></pre><div>And on the S3 side:</div><div><br></div><pre>| Bucket name | Number of objects | Size |</pre><pre>|------------------------------|-------------------|-------------|</pre><pre>| dune-gitlab-artifacts | 667385 | 171.222 GiB |</pre><pre>| dune-gitlab-dependency-proxy | 0 | 0.000 GiB |</pre><pre>| dune-gitlab-packages | 7970 | 44.667 GiB |</pre><pre>| dune-gitlab-registry | 118906 | 170.384 GiB |</pre><pre>|------------------------------|-------------------|-------------|</pre><pre>| ∑ | 794259 | 386.263 GiB |</pre><pre><br></pre><div>We currently have a quota of 2.5 TB.</div><div><br></div><div>I also configured a policy that objects in S3 will only be marked for deletion and only be removed after 90 days. This should be sufficient in case Gitlab deletes data due to a software malfunction.</div><pre><br></pre><div>Some inconsistencies were found (and still exist):</div><div><br></div><div>~7 GiB of CI artifacts on disk, but missing in database (not moved to S3).</div><div><br></div><div>~40 packages in database, but missing from disk.</div><div><br></div><div>Ansgar</div><div><br></div><div>On Thu, 2026-06-11 at 15:34 +0000, Ansgar Burchardt wrote:</div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div>Hi,</div><div><br></div><div>the virtual disk for Gitlab data is fairly full:</div><div><br></div><pre><a href="mailto:root@dune-gitlab">root@dune-gitlab</a>:~# df -h /srv/data</pre><pre>Filesystem Size Used Avail Use% Mounted on</pre><pre>/dev/sdb1 300G 264G 32G 90% /srv/data</pre><pre><br></pre><div>A large chunk is CI job artifacts and packages:</div><div><br></div><pre><a href="mailto:root@dune-gitlab">root@dune-gitlab</a>:~# du -shc /srv/data/gitlab/gitlab-rails/shared/*</pre><pre>157G /srv/data/gitlab/gitlab-rails/shared/artifacts</pre><pre>3.3G /srv/data/gitlab/gitlab-rails/shared/cache</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/ci_secure_files</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/dependency_proxy</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/encrypted_settings</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/external-diffs</pre><pre>2.7G /srv/data/gitlab/gitlab-rails/shared/lfs-objects</pre><pre>47G /srv/data/gitlab/gitlab-rails/shared/packages</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/pages</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/registry</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/terraform_state</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/tmp</pre><pre>209G total</pre><div><br></div><div>For comparison the Git repository data and Gitlab database are much smaller:</div><div><br></div><pre><a href="mailto:root@dune-gitlab">root@dune-gitlab</a>:~# du -shc /srv/data/gitlab/git-data /srv/data/gitlab/postgresql</pre><pre>14G /srv/data/gitlab/git-data</pre><pre>15G /srv/data/gitlab/postgresql</pre><pre>29G total</pre><pre><br></pre><div>Instead of resizing the disk, I would like to migrate some more data to S3-compatible object storage. We already use object storage for the Docker registry.</div><div><br></div><div>I would migrate the following objects:</div><div><br></div><div>1. Job artifacts (CI/CD job artifacts including logs).</div><div>2. Project packages (PyPI, ...)</div><div>3. Dependency proxy (currently unused, just configure it to use object storage).</div><div><br></div><div>I would keep Git LFS objects on the disk as the S3-compat storage has redundancy (to cover disk failure), but no full backups (when something deletes too much). I don't think losing CI/CD job artifacts or packages is too bad, just like with Docker images.</div><div><br></div><div>The migration should be possible while Gitlab runs. It is also possible to migrate back to local storage.</div><div><br></div><div>See the Gitlab documentation for details: <a href="https://docs.gitlab.com/administration/object_storage/">https://docs.gitlab.com/administration/object_storage/</a></div><div><br></div><div>Regards,</div><div>Ansgar</div><div><br></div></blockquote></body></html>