<html><head><style>pre,code,address {
margin: 0px;
}
h1,h2,h3,h4,h5,h6 {
margin-top: 0.2em;
margin-bottom: 0.2em;
}
ol,ul {
margin-top: 0em;
margin-bottom: 0em;
}
blockquote {
margin-top: 0em;
margin-bottom: 0em;
}
</style></head><body><div>Hi,</div><div><br></div><div>the virtual disk for Gitlab data is fairly full:</div><div><br></div><pre><a href="mailto:root@dune-gitlab">root@dune-gitlab</a>:~# df -h /srv/data</pre><pre>Filesystem Size Used Avail Use% Mounted on</pre><pre>/dev/sdb1 300G 264G 32G 90% /srv/data</pre><pre><br></pre><div>A large chunk is CI job artifacts and packages:</div><div><br></div><pre><a href="mailto:root@dune-gitlab">root@dune-gitlab</a>:~# du -shc /srv/data/gitlab/gitlab-rails/shared/*</pre><pre>157G /srv/data/gitlab/gitlab-rails/shared/artifacts</pre><pre>3.3G /srv/data/gitlab/gitlab-rails/shared/cache</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/ci_secure_files</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/dependency_proxy</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/encrypted_settings</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/external-diffs</pre><pre>2.7G /srv/data/gitlab/gitlab-rails/shared/lfs-objects</pre><pre>47G /srv/data/gitlab/gitlab-rails/shared/packages</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/pages</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/registry</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/terraform_state</pre><pre>0 /srv/data/gitlab/gitlab-rails/shared/tmp</pre><pre>209G total</pre><div><br></div><div>For comparison the Git repository data and Gitlab database are much smaller:</div><div><br></div><pre><a href="mailto:root@dune-gitlab">root@dune-gitlab</a>:~# du -shc /srv/data/gitlab/git-data /srv/data/gitlab/postgresql</pre><pre>14G /srv/data/gitlab/git-data</pre><pre>15G /srv/data/gitlab/postgresql</pre><pre>29G total</pre><pre><br></pre><div>Instead of resizing the disk, I would like to migrate some more data to S3-compatible object storage. We already use object storage for the Docker registry.</div><div><br></div><div>I would migrate the following objects:</div><div><br></div><div>1. Job artifacts (CI/CD job artifacts including logs).</div><div>2. Project packages (PyPI, ...)</div><div>3. Dependency proxy (currently unused, just configure it to use object storage).</div><div><br></div><div>I would keep Git LFS objects on the disk as the S3-compat storage has redundancy (to cover disk failure), but no full backups (when something deletes too much). I don't think losing CI/CD job artifacts or packages is too bad, just like with Docker images.</div><div><br></div><div>The migration should be possible while Gitlab runs. It is also possible to migrate back to local storage.</div><div><br></div><div>See the Gitlab documentation for details: <a href="https://docs.gitlab.com/administration/object_storage/">https://docs.gitlab.com/administration/object_storage/</a></div><div><br></div><div>Regards,</div><div>Ansgar</div><div><br></div></body></html>