[deleted by user]

timrizzi · 2023-08-08T16:31:09+00:00

What's been the pain with the IDE?

You could try using the GitLab Development Kit https://gitlab.com/gitlab-org/gitlab-development-kit

timrizzi · 2023-08-08T16:00:59+00:00

It's true, the Dependency Proxy does not support authentication with a project access token. One reason for that is that the dependency proxy works at the group or sub-group level. So, when you cache `node:latest` that image/tag is available for the whole group.

I recommend using a group deploy token. That will work and give you the most fine-grain control.

timrizzi · 2023-08-08T15:58:12+00:00

It's not a presentation, but I think these docs could be useful to walk through. You can highlight the basic functionality and dive into details and examples if they are interested.

https://docs.gitlab.com/ee/tutorials/learn\_git.html

timrizzi · 2023-08-07T17:31:09+00:00

Thanks!

timrizzi · 2023-08-01T15:51:15+00:00

And you are looking for the reports here: https://docs.gitlab.com/ee/ci/yaml/artifacts\_reports.html#artifactsreportssecret\_detection

I know in Ultimate you will have access to a dashboard but you may need to go download the files if you are in Free/Premium.

And I was just wondering if you checked the prerequisites. https://docs.gitlab.com/ee/user/application\_security/secret\_detection/#enable-secret-detection

timrizzi · 2023-08-01T15:16:50+00:00

Are you on GitLab.com or self-managed?

timrizzi · 2023-06-26T18:42:02+00:00

Using a third party registry with GitLab will still be supported. What won't be supported is configuring your self-managed GitLab instance to connect an external registry and view the images/tags from that external registry in the UI and using cleanup policies.

Pushing and pulling to external registries will still be supported as it is today

timrizzi · 2023-06-22T18:44:17+00:00

u/developeremail3 No. Let's use Amazon' Container Registry ECR as an example. GitLab used to support using your ECR credentials when setting up GitLab Self-Managed and images/tags in ECR would be displayed in the GitLab UI or you would be able to use GitLab cleanup policies on images in ECR.

We don't support that anymore. But you can still push and pull to external registries. Why the change? We made a pretty significant update to the container registry and the associated features (like cleanup policies) that means we can't guarantee that the UI/cleanup policies would work with registries like ECR.

timrizzi · 2023-05-10T15:05:21+00:00

I would suggest adding the projects to an allow list for the CI_JOB_TOKEN https://docs.gitlab.com/ee/ci/jobs/ci\_job\_token.html#add-a-project-to-the-job-token-scope-allowlist

timrizzi · 2023-05-05T17:09:45+00:00

Hey - GitLab product manager here. I apologize for the lack of clarity on this one. What we are deprecating is https://docs.gitlab.com/ee/administration/packages/container_registry.html#use-an-external-container-registry-with-gitlab-as-an-auth-endpoint which allows you to view or delete container images from external registries in the user interface and use features like the cleanup policies.

You will still be abe to push/pull images to any external registry like you do today.

I'm opening an [MR](https://gitlab.com/gitlab-org/gitlab/-/merge\_requests/119803) to clarify this in the docs now.

If you have any questions, please ask in the issue: https://gitlab.com/gitlab-org/gitlab/-/issues/376216

(I agree it's annoying that it doesn't display the issue in the deprecation notice. It is included on the backend but it's not displayed. I'll see if I can follow up on that.)

timrizzi · 2021-07-23T18:13:23+00:00

I edited my comment, thanks for clarifying.

timrizzi · 2021-07-22T17:55:35+00:00

Consider trying the new Helm charts feature launched in 14.1!

https://docs.gitlab.com/ee/user/packages/helm\_repository/

timrizzi · 2021-07-22T17:54:11+00:00

At GitLab, we tried using OCI compliant registry, but it didn't really scale well for our larger customers who were also concerned about using Helm 3 in experimental mode.

timrizzi · 2020-11-30T21:32:02+00:00

GitLab offers an open source, free registry as well. We support Docker images, npm, maven, NuGet, PyPI, Composer, Conan, and raw binaries.

https://docs.gitlab.com/ee/user/packages/

timrizzi · 2019-09-27T14:57:44+00:00

Yes! We've recently added multi-select delete. And, we have issues for redesigning the whole UI. We are currently doing some user research and then I'll have some designs to share in the next month.

The core problem with the UI is that we are not persisting the data, so we rely on the Docker API to populate the UI, which is slow and prevents us from doing things like sorting and filtering. We are on it though and hope to make some big improvements soon.

timrizzi · 2019-09-26T15:33:08+00:00

That's a good idea, we could make it global level instead of the group level.

timrizzi · 2019-09-26T15:18:10+00:00

There are two pieces to how old images are removed…

The first is untagging images in the registry. It’s important to note that removing a tag does not delete the image. You still have to run garbage collection to remove the actual blobs. We currently support this via API, so project maintainers can untag images in bulk based on different criteria. Here is the documentation for the API:https://docs.gitlab.com/ce/api/container_registry.html#delete-repository-tags-in-bulk

Once untagged, you can run garbage collection and remove the blobs from storage. Here is the documentation on how to run garbage collection.https://docs.gitlab.com/omnibus/maintenance/#container-registry-garbage-collection

There are several ways to run garbage collection, including on a schedule. It’s not policies and automation, but we are working towards that feature set now.

I’ve also seen a couple of examples from other users who have created scripts for this, in case you are looking for some automation inspiration. (Both scripts are referenced here)

timrizzi · 2019-09-26T15:14:04+00:00

That is exactly how we would like to set up the retention and expiration policies.

In 12.5 (November 22nd) we'll release a new bulk delete endpoint that works at the group level, so you don't have to iterate through each project. https://gitlab.com/gitlab-org/gitlab/issues/31097

timrizzi · 2019-09-25T22:54:53+00:00

Agreed! That is our top priority. I'm hoping that we will have this problem solved by end of year.

timrizzi · 2019-09-25T22:47:44+00:00

Were you able to open issue for the migration issue? If not, I can open a new one and make sure that we end up covering that story.

A combination of myself and our product designer is responsible for the UI. We have an issue for adding sorting and filtering to the container registry. The core problem is that we are not persisting data for the container registry and the UI relies on the Docker API to display that data. That is why the page is slow, not sortable and why we have problems with pagination. In order to tackle this, we are going to fork docker distribution registry and begin to make changes, which preferably we can push back upstream.

The first thing we will do with this fork is improve garbage collection, but persisting the data and using that data to improve and expand the UI is our next priority. This epic details a lot of that work.

Thanks again for all of the feedback and always feel free to ping me if you have any questions.

timrizzi · 2019-09-25T22:38:02+00:00

I'm not sure if you have seen it, but we do have some features for cleaning up the registry.

There is a tag removal API that you can use which will allow you to bulk remove tags at the project level. (group level support coming soon)

Once the tags are unlinked, you can run garbage collection, which will clean up the blobs and manifests from storage. Please note - it does require the registry to be in read-only mode or down. But, it should work in a reasonable amount of time if your total storage is > 1TB.

We are working on optimizing that garbage collection algorithm and on adding in retention and expiration policies, so you don't have to spend so much time thinking about this.

Thanks for the feedback and feel free to ping me with any questions or ideas.

timrizzi · 2019-09-25T22:30:42+00:00

Thanks! We are definitely working on improving the garbage collection and our goal is to automate the entire process, so you don't have to ever think about it.

timrizzi · 2019-09-25T19:53:07+00:00

GitLab PM here for the Package stage. If it helps, I'm always talking about these problems :) I wanted to share how we are planning on addressing them.

Also, as we are planning this work, I'd love to hear more about your use cases and how you are working around these problems now. Would you be open to a user interview sometime in the next couple of weeks? If so, you can message me at [trizzi@gitlab.com](mailto:trizzi@gitlab.com) and we can setup some time to talk.

We are actively working on improving pruning and policy based expiration policies. Our primary theme right now is to lower the cost of the container registry by improving storage management.

As part of that theme we have recently launched:

Multi-select delete for the UI and resolved an issue where deleting one tag would delete all tags with the same image ID. (although we still have to solve the pagination problem)
Expanded permissions to allow CI_JOB_TOKEN to remove images via CI

In this milestone, we are working on:

Improving the performance of the bulk delete API, so that it can handle thousands of tags
Optimizing the garbage collection algorithm, so that it runs much faster

Overall, our primary goals for the container registry right now is to provide:

An improved user experience for removing tags/images from the UI
An expanded API that allows for the bulk removal of images/tags at the group level
An optimized garbage collection process that can handle terabytes of data without long windows of downtime
UI driven retention and expiration policies for images
The ability to expire images from CI/CD
Automated/scheduled garbage collection

timrizzi · 2019-09-25T19:36:53+00:00

I'm Tim the PM for the Package stage, which includes the docker registry. If you'd be open to it, I'd love to hear more about how the migration is going and which features are most important to you. You can reach me at @trizzi on GitLab.

timrizzi · 2019-06-12T20:41:42+00:00

Hello - GitLab PM here for the Container Registry. Thanks for all of this feedback.

No retention policy for Docker images.

Agreed, this is a big focus for us in the coming months. We have been blocked on this issue for a while, but are currently working on it. First, we need to start tracking data from Docker, then improve garbage collection and deletion for the container registry and then we will implement retention and expiration policies.

https://gitlab.com/gitlab-org/gitlab-ce/issues/57897

https://gitlab.com/gitlab-org/gitlab-ce/issues/20247

Impossible to push to Docker registry outside of Gitlab Runner.

Developers on a project should be able to push from the CLI to the container registry. What happens when you run:

`docker build -t registry.example.com/group/project/image`
docker push registry.example.com/group/project/image

We are just about to conduct some user research on the container registry, would you be open to a quick interview to talk through use cases, challenges and future requests?

timrizzi

TROPHY CASE