📄 Scheduling Apache Hop tasks on Windows, Linux, and Jenkins

Scheduling Apache Hop tasks on Windows, Linux, and Jenkins

Last updated: February 11, 2026

This article outlines how to schedule Apache Hop tasks on different platforms, Windows, Linux, and Jenkins. These methods allow you to automate the execution of your Apache Hop workflows and pipelines, ensuring they run on a set schedule without manual intervention.

Scheduling Tasks on Windows

Prerequisites

Before setting up scheduled tasks on Windows, ensure that you have a working hop-run.bat command to execute a workflow or pipeline within a project and environment. This command should also include any required parameters. A sample command may look like this:

bash

hop-run.bat -p project_name -e environment_name -f workflow_name.hpl -lf c:\hop\hop-logs\project_name.log

For more detailed information, refer to the Apache Hop documentation.

Steps to Schedule Apache Hop Tasks Using Windows Task Scheduler

  1. Open Windows Task Scheduler

    Navigate to Start > Windows Administrative Tools > Task Scheduler.

  2. Create a Basic Task

    • Select Create Basic Task from the right-hand menu.

    • Provide a name for the task, such as "Run Apache Hop Pipeline".

    • Optionally, you can also add a description for the task.

  3. Set the Trigger

    • In the Trigger section, specify when you want the task to run (e.g., daily, weekly, or every hour).

    • For example, you might choose "Daily" and set it to run every day at 9 AM, or choose "Repeat every 1 hour" to run every hour.

  4. Set the Action

    • In the Action section, select Start a Program.

    • Browse to your hop-run.bat file and add the required arguments in the Add arguments (optional) field.

      For example:

bash

-p project_name -e environment_name -f workflow_name.hpl -lf c:\hop\hop-logs\project_name.log
  1. Finish the Setup

    • Review the task settings and click Finish. Your task will now run according to the schedule you've set.

Note: To append a date/time to the log file, you can add custom date formatting to the arguments in the batch file. Consult Apache Hop documentation for more details.

Scheduling Tasks on Linux Using Cron Jobs

Prerequisites

Before setting up cron jobs to schedule Apache Hop tasks, make sure you have a working docker run command for your Apache Hop container. You should also have a wrapper script that helps manage the scheduling process. A sample command could look like this:

bash

#!/bin/bash
docker run --rm \
--env HOP_LOG_LEVEL=Basic \
--env HOP_FILE_PATH='${PROJECT_HOME}/code/process-data.hwf' \
--env HOP_PROJECT_FOLDER=/files \
--env HOP_PROJECT_NAME=project_name \
--env HOP_ENVIRONMENT_CONFIG_FILE_NAME_PATHS=/config/config-file1.json,/config/config-file2.json,/config/config-file3.json,/config/config-file4.json \
--env HOP_RUN_CONFIG=local \
--name hop-project_name \
-v /home/hop/project-files:/files \
-v /home/hop/project-config:/config \
-v /opt/project-data:/opt/project-data:rw \
-m 2g \
apache/hop:Development > $HOME/logs/project_name/project_name-$(date --utc +\%Y\%m\%dT\%H\%M\%SZ).log

Steps to Schedule Apache Hop Tasks with Cron on Linux

Create a Wrapper Script

Create a new shell script to wrap your docker run command. For example:

bash

#!/bin/bash
docker run --rm \
--env HOP_LOG_LEVEL=Basic \
--env HOP_FILE_PATH='${PROJECT_HOME}/code/process-data.hwf' \
--env HOP_PROJECT_FOLDER=/files \
--env HOP_PROJECT_NAME=project_name \
--env HOP_ENVIRONMENT_CONFIG_FILE_NAME_PATHS=/config/config-file1.json,/config/config-file2.json,/config/config-file3.json,/config/config-file4.json \
--env HOP_RUN_CONFIG=local \
--name hop-project_name \
-v /home/hop/project-files:/files \
-v /home/hop/project-config:/config \
-v /opt/project-data:/opt/project-data:rw \
-m 2g \
apache/hop:Development > $HOME/logs/project_name/project_name-$(date --utc +\%Y\%m\%dT\%H\%M\%SZ).log
  1. Save this script, e.g., as /home/hop/run-project_name.sh.

Make the Script Executable

Run the following command to make the script executable:

bash

chmod +x /home/hop/run-project_name.sh

Add the Cron Job

Open the crontab file for editing:

crontab -e

Add the following line to schedule the script:

m h dom mon dow command
15 * * * * sh /home/hop/run-project_name.sh
  1. This cron job will execute the run-project_name.sh script every hour, at the 15th minute.

Scheduling Apache Hop Tasks in Jenkins

Prerequisites

Before configuring Jenkins to schedule Apache Hop tasks, check the following:

  • You have a Jenkins instance running.

  • The hop-run.bat (for Windows) or docker run (for Linux/macOS) command is functional and tested.

Steps to Schedule Apache Hop Tasks with Jenkins

1. Create a New Jenkins Job

  1. Log in to your Jenkins instance.

  2. Click on New Item and create a new Freestyle Project.

  3. Name your project appropriately (e.g., "Apache Hop Data Pipeline").

2. Configure Source Control (Optional)

If your Apache Hop workflow is stored in a version control system (e.g., Git), configure Jenkins to pull the latest version by setting up the appropriate source control repository under Source Code Management.

3. Configure the Build Step

In the Build section, add a build step to execute the hop-run.bat (for Windows) or docker run (for Linux/macOS) command.

For Windows (Batch Command): Use the Execute Windows batch command option:

bash

hop-run.bat -p project_name -e environment_name -f workflow_name.hpl -lf c:\hop\hop-logs\project_name.log

For Linux/macOS (Shell Command): Use the Execute shell option:

bash

/home/hop/run-project_name.sh
  • For macOS, the shell script will typically be the same as Linux, but the path to the project files or environment may differ (e.g., /Users/username/project_name).

4. Configure Scheduling

In the Build Triggers section, enable Build periodically. Use cron syntax to specify the schedule for your Apache Hop task. For example, to run the job at the 15th minute of every hour, use:

H 15 * * *  # Runs at the 15th minute of every hour

You can customize the cron syntax based on your requirements:

  • H 0 * * * - Runs at midnight every day.

  • H 0 * * 1-5 - Runs at midnight, Monday through Friday.

5. Save and Trigger the Job

Once the configuration is complete, click Save. Jenkins will automatically trigger the Apache Hop task according to the specified schedule.