Alternative directory implementation of ArtifactManager?

[Staffan Forsell]

Hello,
I have been looking for a way moving the artifacts out of the builds
directory and have been looking at the ArtifactManager.
Is there a reason that I can't find an implementation of this that just
uses a different local directory on the master?
Would should an plugin be hard to implement and how would one start?
Is there any obvious reasons that just creating a system property that
switches the artifacts to a different dir would not work?

Our use-case is that we create a lot of artifacts (zips/vms etc) that we
want to save for testing. We do this by changing the whole buildsDir to a
glusterfs. This gives us massive amounts of space but the less than ideal
performance with many small files, so we would like to separate the rest of
the jobs and build files from the artifacts.

Best regards Staffan

[Markus Winter]

Implementation is quite easy I would say.
You need an ArtifactManager (look at the StandardArtifactManager, you
can probably just copy and replace the getArtifactsDir() method)
And an ArtifactManagerFactory where you can configure the root path for
the artifacts

Post by Staffan Forsell
Hello,
I have been looking for a way moving the artifacts out of the builds
directory and have been looking at the ArtifactManager.
Is there a reason that I can't find an implementation of this that
just uses a different local directory on the master?
Would should an plugin be hard to implement and how would one start?
Is there any obvious reasons that just creating a system property that
switches the artifacts to a different dir would not work?
Our use-case is that we create a lot of artifacts (zips/vms etc) that
we want to save for testing. We do this by changing the whole
buildsDir to a glusterfs. This gives us massive amounts of space but
the less than ideal performance with many small files, so we would
like to separate the rest of the jobs and build files from the artifacts.
Best regards Staffan
--
You received this message because you are subscribed to the Google
Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-dev/00cbdc00-ed36-4a4d-9187-4af486e44822%40googlegroups.com
<https://groups.google.com/d/msgid/jenkinsci-dev/00cbdc00-ed36-4a4d-9187-4af486e44822%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[Daniel Beck]

Is there any obvious reasons that just creating a system property that switches the artifacts to a different dir would not work?

Given how many regressions the custom builds dir introduced due to moving jobs between folders, renaming them, etc., and all those actions newly having to be mirrored in a different structure, I'm not surprised nobody bothered -- just having custom builds dir is good enough in many cases. Would not recommend attempting it with using job names for the lookup.

Would look at artifact-manager-s3 as a sort of template for custom implementation for artifact storage.

[Staffan Forsell]

artifact-manager-s3 seems to contain examples of most of the things needed.
I did not quite understand...do you recommend against trying to do this or
something else? If we would implement the same tests and use the same key
as artifact-manager-s3, I would thing it would not be unreasonable work?

I figure if there is infrastructure now for this to work on s3 or azure,
then just mapping it to a different directory should be pretty simple in my
mind.

Another way of doing this would be to just invest in lots of SSD storage
for our jenkins servers... Might be cheaper if this would prove to be very
complicated.

Post by Staffan Forsell

Post by Staffan Forsell
Is there any obvious reasons that just creating a system property that

switches the artifacts to a different dir would not work?
Given how many regressions the custom builds dir introduced due to moving
jobs between folders, renaming them, etc., and all those actions newly
having to be mirrored in a different structure, I'm not surprised nobody
bothered -- just having custom builds dir is good enough in many cases.
Would not recommend attempting it with using job names for the lookup.
Would look at artifact-manager-s3 as a sort of template for custom
implementation for artifact storage.

[Jesse Glick]

Another way of doing this would be to just invest in lots of SSD storage for our jenkins servers.

Or just changing your projects to deploy artifacts to a proper
repository manager. The `archiveArtifacts` step and its kin is
intended only as a simple built-in way to retain smallish files if you
have not yet bothered to set up a real system.

[Staffan Forsell]

Well, we have lots of jobs, both freestyle and different styles of the new pipeline, not all for sw that's currently in active development but we still need to know we can build these.
We would rather invest our time in something that's useful for all jobs without changing all those old job definitions.
Also everything that we keep for a while on Jenkins as an artifact is not necessarily something that we would deploy to artifactory. Coupling the artifact retention to the number of builds in Jenkins is often easier than having a "works for all cleanup policy" in artifactory.

[Jesse Glick]

Post by Staffan Forsell
artifact-manager-s3 seems to contain examples of most of the things needed.

Too complicated for your purpose. Creating a plugin with a lightly
edited clone of `StandardArtifactManager` would suffice. (Use the
Maven archetype for a plugin with global configuration, so the user
can pick the location in the GUI or using configuration-as-code.)