Running stages in parallel with Jenkins workflow pipeline

Question

Please note  the question is based on the old  now called  scripted  pipeline format  When using  declarative pipelines   parallel blocks can be nested inside of stage blocks  see Parallel stages with Declarative Pipeline 1 2     I m wondering how parallel steps are supposed to work with Jenkins workflow pipeline plugin  esp  how to mix them with build stages  I know about the general pattern   parallel firstTask         Do some stuff    secondTask         Do some other stuff in parallel      However  I d like to run couple of stages in parallel  on the same node  which has multiple executors   so I tried to add stages like this   stage  A     Do some preparation stuff  parallel firstTask      stage  B1       Do some stuff    secondTask      stage  B2       Do some other stuff in parallel     stage  C     Finalizing stuff   This does not work as expected  The  do stuff  tasks are executed in parallel  but the parallel stages end immediately and do not incorporate the stuff they should contain  As a consequence  the Stage View does not show the correct result and also does not link the logs   Can I build different stages in parallel  or is the  parallel  step only meant to be used within a single stage

User · Accepted Answer

You may not place the deprecated non-block-scoped stage (as in the original question) inside parallel.

As of JENKINS-26107, stage takes a block argument. You may put parallel inside stage or stage inside parallel or stage inside stage etc. However visualizations of the build are not guaranteed to support all nestings; in particular

The built-in Pipeline Steps (a “tree table” listing every step run by the build) shows arbitrary stage nesting.
The Pipeline Stage View plugin will currently only display a linear list of stages, in the order they started, regardless of nesting structure.
Blue Ocean will display top-level stages, plus parallel branches inside a top-level stage, but currently no more.

JENKINS-27394, if implemented, would display arbitrarily nested stages.

User · Answer

I have just tested the following pipeline and it works  parallel firstBranch        stage   Starting Test                  build job   test1   parameters   string name   Environment   value    env Environment             secondBranch        stage   Starting Test2                  build job   test2   parameters   string name   Environment   value    env Environment              This Job named  trigger-test  accepts one parameter named  Environment   Job  test1  and  test2  are simple jobs   Example for  test1    One parameter named  Environment  Pipeline   echo   env Environment-TEST1    On execution  I am able to see both stages running in the same time

User · Answer

I think this has been officially implemented now  https   jenkins io blog 2017 09 25 declarative-1

User · Answer

As  Quartz mentioned  you can do something like  stage  Tests         parallel           Unit Tests                 container  node                     sh  npm test --cat unit                                     API Tests                 container  node                     sh  npm test --cat acceptance

User · Answer

that syntax is now deprecated  you will get this error   org codehaus groovy control MultipleCompilationErrorsException  startup failed  WorkflowScript  14  Expected a stage   line 14  column 9         parallel firstTask              WorkflowScript  14  Stage does not have a name   line 14  column 9         parallel secondTask              2 errors   You should do something like   stage  Parallel         steps           parallel                firstTask                        do some stuff                             secondTask                         Do some other stuff in parallel                                   Just to add the use of node here  to distribute jobs across multiple build servers  VMs   pipeline     stages       stage  Work 1         steps        parallel    Build common Library                                  node   lt Label gt                           your stuff                                              Build Utilities                  node   lt Label gt                        your stuff                                                   All VMs should be labelled as  to use as a pool

[jenkins] Running stages in parallel with Jenkins workflow / pipeline

Examples related to jenkins

Examples related to jenkins-pipeline