Чем отличаются scripted и declarative pipeline в jenkins

If you read this blog post, there is a high chance you’re looking for information about practical differences between scripted and declarative pipeline, correct? You couldn’t find a better place then. I’m going to show you the four most practical differences between those two. Stay with me for a few minutes and enjoy the ride!

The introduction

If you ask me this question and expect an answer different from «it depends,» I would say use the declarative pipeline. And here’s why.

1. Pipeline code validation at startup

Let’s consider the following pipeline code.

If we try to execute the following pipeline, the validation will quickly fail the build. The echo step can be triggered only with the String parameter, so we get the error like.

Now let’s take a look at the scripted pipeline equivalent of that example.

This pipeline executes the same stages, and the same steps. There is one significant difference however. Let’s execute it and see what result it produces.

It failed as expected. But this time the Build stage was executed, as well as the first step from the Test stage. As you can see, there was no validation of the pipeline code. The declarative pipeline in this case handles such use case much better.

2. Restart from stage

Another cool feature that only declarative pipeline has is «Restart from stage» one. Let’s fix the pipeline from the previous example and see if we can restart the Test stage only.

Источник

Конвейер со сценариями Jenkins или декларативный конвейер

Я не уверен, в какой ситуации каждый из этих двух типов лучше всего подходит. Так будет declarative ли будущее трубопровода Дженкинса?

Любой, кто может поделиться мыслями об этих двух типах синтаксиса.

При первом создании Jenkins Pipeline в качестве основы был выбран Groovy. Jenkins уже давно поставляет со встроенным движком Groovy, чтобы предоставить расширенные возможности создания сценариев как для администраторов, так и для пользователей. Вдобавок разработчики Jenkins Pipeline обнаружили, что Groovy является прочной основой для построения того, что теперь называется DSL «Scripted Pipeline».

Поскольку это полнофункциональная среда программирования, Scripted Pipeline предлагает пользователям Jenkins огромную гибкость и расширяемость. Кривая обучения Groovy обычно не желательна для всех членов данной команды, поэтому был создан Declarative Pipeline, предлагающий более простой и упорядоченный синтаксис для разработки Jenkins Pipeline.

По сути, обе эти подсистемы представляют собой одну и ту же подсистему трубопроводов. Оба они являются надежными реализациями «конвейера как кода». Оба они могут использовать шаги, встроенные в Pipeline или предоставляемые плагинами. Оба могут использовать общие библиотеки.

Однако они отличаются синтаксисом и гибкостью. Декларативная ограничивает то, что доступно пользователю, с более строгой и заранее определенной структурой, что делает его идеальным выбором для более простых конвейеров непрерывной доставки. Scripted предоставляет очень мало ограничений, поскольку единственные ограничения по структуре и синтаксису, как правило, определяются самим Groovy, а не какими-либо специфичными для конвейера системами, что делает его идеальным выбором для опытных пользователей и тех, у кого более сложные требования. Как следует из названия, Declarative Pipeline поощряет модель декларативного программирования. В то время как сценарии конвейеров следуют более императивной модели программирования.

Источник

Pipeline

This chapter covers all recommended aspects of Jenkins Pipeline functionality, including how to:

use different development tools to facilitate the creation of your Pipeline, and

For an overview of content in the Jenkins User Handbook, see User Handbook overview.

What is Jenkins Pipeline?

Jenkins Pipeline (or simply «Pipeline» with a capital «P») is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.

A continuous delivery (CD) pipeline is an automated expression of your process for getting software from version control right through to your users and customers. Every change to your software (committed in source control) goes through a complex process on its way to being released. This process involves building the software in a reliable and repeatable manner, as well as progressing the built software (called a «build») through multiple stages of testing and deployment.

Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines «as code» via the Pipeline domain-specific language (DSL) syntax. [1]

The definition of a Jenkins Pipeline is written into a text file (called a Jenkinsfile ) which in turn can be committed to a project’s source control repository. [2] This is the foundation of «Pipeline-as-code»; treating the CD pipeline a part of the application to be versioned and reviewed like any other code.

Creating a Jenkinsfile and committing it to source control provides a number of immediate benefits:

Automatically creates a Pipeline build process for all branches and pull requests.

Code review/iteration on the Pipeline (along with the remaining source code).

Audit trail for the Pipeline.

Single source of truth [3] for the Pipeline, which can be viewed and edited by multiple members of the project.

While the syntax for defining a Pipeline, either in the web UI or with a Jenkinsfile is the same, it is generally considered best practice to define the Pipeline in a Jenkinsfile and check that in to source control.

Declarative versus Scripted Pipeline syntax

Declarative and Scripted Pipelines are constructed fundamentally differently. Declarative Pipeline is a more recent feature of Jenkins Pipeline which:

provides richer syntactical features over Scripted Pipeline syntax, and

is designed to make writing and reading Pipeline code easier.

Why Pipeline?

Jenkins is, fundamentally, an automation engine which supports a number of automation patterns. Pipeline adds a powerful set of automation tools onto Jenkins, supporting use cases that span from simple continuous integration to comprehensive CD pipelines. By modeling a series of related tasks, users can take advantage of the many features of Pipeline:

Code: Pipelines are implemented in code and typically checked into source control, giving teams the ability to edit, review, and iterate upon their delivery pipeline.

Durable: Pipelines can survive both planned and unplanned restarts of the Jenkins controller.

Pausable: Pipelines can optionally stop and wait for human input or approval before continuing the Pipeline run.

Versatile: Pipelines support complex real-world CD requirements, including the ability to fork/join, loop, and perform work in parallel.

Extensible: The Pipeline plugin supports custom extensions to its DSL [1] and multiple options for integration with other plugins.

While Jenkins has always allowed rudimentary forms of chaining Freestyle Jobs together to perform sequential tasks, [4] Pipeline makes this concept a first-class citizen in Jenkins.

Building on the core Jenkins value of extensibility, Pipeline is also extensible both by users with Pipeline Shared Libraries and by plugin developers. [5]

The flowchart below is an example of one CD scenario easily modeled in Jenkins Pipeline:

Pipeline concepts

The following concepts are key aspects of Jenkins Pipeline, which tie in closely to Pipeline syntax (see the overview below).

Pipeline

A Pipeline is a user-defined model of a CD pipeline. A Pipeline’s code defines your entire build process, which typically includes stages for building an application, testing it and then delivering it.

A node is a machine which is part of the Jenkins environment and is capable of executing a Pipeline.

Stage

A stage block defines a conceptually distinct subset of tasks performed through the entire Pipeline (e.g. «Build», «Test» and «Deploy» stages), which is used by many plugins to visualize or present Jenkins Pipeline status/progress. [6]

Pipeline syntax overview

The following Pipeline code skeletons illustrate the fundamental differences between Declarative Pipeline syntax and Scripted Pipeline syntax.

Be aware that both stages and steps (above) are common elements of both Declarative and Scripted Pipeline syntax.

Declarative Pipeline fundamentals

In Declarative Pipeline syntax, the pipeline block defines all the work done throughout your entire Pipeline.

Scripted Pipeline fundamentals

In Scripted Pipeline syntax, one or more node blocks do the core work throughout the entire Pipeline. Although this is not a mandatory requirement of Scripted Pipeline syntax, confining your Pipeline’s work inside of a node block does two things:

Schedules the steps contained within the block to run by adding an item to the Jenkins queue. As soon as an executor is free on a node, the steps will run.

Creates a workspace (a directory specific to that particular Pipeline) where work can be done on files checked out from source control.
Caution: Depending on your Jenkins configuration, some workspaces may not get automatically cleaned up after a period of inactivity. See tickets and discussion linked from JENKINS-2111 for more information.

Pipeline example

Read more about Pipeline syntax on the Pipeline Syntax page.

Источник

Jenkins Pipeline: заметки об оптимизации. Часть 1

Меня зовут Илья Гуляев, я занимаюсь автоматизацией тестирования в команде Post Deployment Verification в компании DINS.

В DINS мы используем Jenkins во многих процессах: от сборки билдов до запуска деплоев и автотестов. В моей команде мы используем Jenkins в качестве платформы для единообразного запуска смоук-проверок после деплоя каждого нашего сервиса от девелоперских окружений до продакшена.

Год назад другие команды решили использовать наши пайплайны не только для проверки одного сервиса после его обновления, но и проверять состояние всего окружения перед запуском больших пачек тестов. Нагрузка на нашу платформу возросла в десятки раз, и Jenkins перестал справляться с поставленной задачей и стал просто падать. Мы быстро поняли, что добавление ресурсов и тюнинг сборщика мусора могут лишь отсрочить проблему, но не решат ее полностью. Поэтому мы решили найти узкие места Jenkins и оптимизировать их.

В этой статье я расскажу, как работает Jenkins Pipeline, и поделюсь своими находками, которые, возможно, помогут вам сделать пайплайны быстрее. Материал будет полезен инженерам, которые уже работали с Jenkins, и хотят познакомиться с инструментом ближе.

Что за зверь Jenkins Pipeline

Jenkins Pipeline — мощный инструмент, который позволяет автоматизировать различные процессы. Jenkins Pipeline представляет собой набор плагинов, которые позволяют описывать действия в виде Groovy DSL, и является преемником плагина Build Flow.

Скрипт для плагина Build Flow исполнялся напрямую на мастере в отдельном Java-потоке, который выполнял Groovy-код без барьеров, препятствующих доступу к внутреннему API Jenkins. Данный подход представлял угрозу безопасности, что впоследствии стало одной из причин отказа от Build Flow, и послужило предпосылкой для создания безопасного и масштабируемого инструмента для запуска скриптов — Jenkins Pipeline.

Подробнее об истории создания Jenkins Pipeline можно узнать из статьи автора Build Flow или доклада Олега Ненашева о Groovy DSL в Jenkins.

Как работает Jenkins Pipeline

Теперь разберемся, как работают пайплайны изнутри. Обычно говорят, что Jenkins Pipeline — совершенно другой вид заданий в Jenkins, непохожий на старые добрые freestyle-джобы, которые можно накликать в веб-интерфейсе. С точки зрения пользователя это, может, и выглядит так, но со стороны Jenkins пайплайны — набор плагинов, которые позволяют перенести описание действий в код.

Сходства Pipeline и Freestyle джобы

Правда в том, что параметры, описанные в Pipeline, автоматически добавятся в раздел настройки в веб-интерфейсе при запуске джобы. Верить мне можно потому, что этот функционал в последней редакции писал я, но об этом подробнее во второй части статьи.

Отличия Pipeline и Freestyle джобы

Запуск Jenkins Declarative Pipeline

Процесс запуска Jenkins Pipeline состоит из следующих шагов:

При старте pipeline-джобы Jenkins создает отдельный поток и направляет задание в очередь на выполнение, а после загрузки скрипта определяет, какой агент нужен для выполнения задачи.

Для поддержки данного подхода используется специальный пул потоков Jenkins (легковесных исполнителей). Можно заметить, что они выполняются на мастере, но не затрагивают обычный пул исполнителей:

Количество потоков в данном пуле не ограничено (на момент написания статьи).

Работа параметров в Pipeline. А также триггеров и некоторых опций

Обработку параметров можно описать формулой:

Из параметров джобы, которые мы видим при запуске, сначала удаляются параметры Pipeline из предыдущего запуска, а уже потом добавляются параметры заданные в Pipeline текущего запуска. Это позволяет удалить параметры из джобы, если они были удалены из Pipeline.

Источник

Getting started with Pipeline

As mentioned previously, Jenkins Pipeline is a suite of plugins that supports implementing and integrating continuous delivery pipelines into Jenkins. Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines «as code» via the Pipeline DSL. [1]

This section describes how to get started with creating your Pipeline project in Jenkins and introduces you to the various ways that a Jenkinsfile can be created and stored.

Prerequisites

To use Jenkins Pipeline, you will need:

Jenkins 2.x or later (older versions back to 1.642.3 may work but are not recommended)

Pipeline plugin, [2] which is installed as part of the «suggested plugins» (specified when running through the Post-installation setup wizard after installing Jenkins).

Read more about how to install and manage plugins in Managing Plugins.

Defining a Pipeline

Both Declarative and Scripted Pipeline are DSLs [1] to describe portions of your software delivery pipeline. Scripted Pipeline is written in a limited form of Groovy syntax.

Relevant components of Groovy syntax will be introduced as required throughout this documentation, so while an understanding of Groovy is helpful, it is not required to work with Pipeline.

A Pipeline can be created in one of the following ways:

The syntax for defining a Pipeline with either approach is the same, but while Jenkins supports entering Pipeline directly into the classic UI, it is generally considered best practice to define the Pipeline in a Jenkinsfile which Jenkins will then load directly from source control.

Through Blue Ocean

If you are new to Jenkins Pipeline, the Blue Ocean UI helps you set up your Pipeline project, and automatically creates and writes your Pipeline (i.e. the Jenkinsfile ) for you through the graphical Pipeline editor.

As part of setting up your Pipeline project in Blue Ocean, Jenkins configures a secure and appropriately authenticated connection to your project’s source control repository. Therefore, any changes you make to the Jenkinsfile via Blue Ocean’s Pipeline editor are automatically saved and committed to source control.

Read more about Blue Ocean in the Blue Ocean chapter and Getting started with Blue Ocean page.

Through the classic UI

A Jenkinsfile created using the classic UI is stored by Jenkins itself (within the Jenkins home directory).

To create a basic Pipeline through the Jenkins classic UI:

If required, ensure you are logged in to Jenkins.

From the Jenkins home page (i.e. the Dashboard of the Jenkins classic UI), click New Item at the top left.

In the Enter an item name field, specify the name for your new Pipeline project.
Caution: Jenkins uses this item name to create directories on disk. It is recommended to avoid using spaces in item names, since doing so may uncover bugs in scripts that do not properly handle spaces in directory paths.

Scroll down and click Pipeline, then click OK at the end of the page to open the Pipeline configuration page (whose General tab is selected).

Click the Pipeline tab at the top of the page to scroll down to the Pipeline section.
Note: If instead you are defining your Jenkinsfile in source control, follow the instructions in In SCM below.

In the Pipeline section, ensure that the Definition field indicates the Pipeline script option.

Enter your Pipeline code into the Script text area.
For instance, copy the following Declarative example Pipeline code (below the Jenkinsfile ( …​ ) heading) or its Scripted version equivalent and paste this into the Script text area. (The Declarative example below is used throughout the remainder of this procedure.)

Note: You can also select from canned Scripted Pipeline examples from the try sample Pipeline option at the top right of the Script text area. Be aware that there are no canned Declarative Pipeline examples available from this field.

Click Save to open the Pipeline project/item view page.

On this page, click Build Now on the left to run the Pipeline.

Under Build History on the left, click #1 to access the details for this particular Pipeline run.

Click Console Output to see the full output from the Pipeline run. The following output shows a successful run of your Pipeline.

Notes:

You can also access the console output directly from the Dashboard by clicking the colored globe to the left of the build number (e.g. #1).

In SCM

Complex Pipelines are difficult to write and maintain within the classic UI’s Script text area of the Pipeline configuration page.

To make this easier, your Pipeline’s Jenkinsfile can be written in a text editor or integrated development environment (IDE) and committed to source control [3] (optionally with the application code that Jenkins will build). Jenkins can then check out your Jenkinsfile from source control as part of your Pipeline project’s build process and then proceed to execute your Pipeline.

To configure your Pipeline project to use a Jenkinsfile from source control:

Follow the procedure above for defining your Pipeline through the classic UI until you reach step 5 (accessing the Pipeline section on the Pipeline configuration page).

From the Definition field, choose the Pipeline script from SCM option.

Complete the fields specific to your repository’s source control system.
Tip: If you are uncertain of what value to specify for a given field, click its ? icon to the right for more information.

When you update the designated repository, a new build is triggered, as long as the Pipeline is configured with an SCM polling trigger.

Built-in Documentation

Pipeline ships with built-in documentation features to make it easier to create Pipelines of varying complexities. This built-in documentation is automatically generated and updated based on the plugins installed in the Jenkins instance.

Snippet Generator

The built-in «Snippet Generator» utility is helpful for creating bits of code for individual steps, discovering new steps provided by plugins, or experimenting with different parameters for a particular step.

The Snippet Generator is dynamically populated with a list of the steps available to the Jenkins instance. The number of steps available is dependent on the plugins installed which explicitly expose steps for use in Pipeline.

To generate a step snippet with the Snippet Generator:

Select the desired step in the Sample Step dropdown menu

Use the dynamically populated area below the Sample Step dropdown to configure the selected step.

Click Generate Pipeline Script to create a snippet of Pipeline which can be copied and pasted into a Pipeline.

To access additional information and/or documentation about the step selected, click on the help icon (indicated by the red arrow in the image above).

Global Variable Reference

In addition to the Snippet Generator, which only surfaces steps, Pipeline also provides a built-in «Global Variable Reference.» Like the Snippet Generator, it is also dynamically populated by plugins. Unlike the Snippet Generator however, the Global Variable Reference only contains documentation for variables provided by Pipeline or plugins, which are available for Pipelines.

The variables provided by default in Pipeline are:

Declarative Directive Generator

While the Snippet Generator helps with generating steps for a Scripted Pipeline or for the steps block in a stage in a Declarative Pipeline, it does not cover the sections and directives used to define a Declarative Pipeline. The «Declarative Directive Generator» utility helps with that. Similar to the Snippet Generator, the Directive Generator allows you to choose a Declarative directive, configure it in a form, and generate the configuration for that directive, which you can then use in your Declarative Pipeline.

To generate a Declarative directive using the Declarative Directive Generator:

Select the desired directive in the dropdown menu

Use the dynamically populated area below the dropdown to configure the selected directive.

Click Generate Directive to create the directive’s configuration to copy into your Pipeline.

Источник