JobEngine
Overview
The JobEngine manages all regular and scheduled job tasks within the merlot.aero application suite. It is a "worker role" that is responsible for executing most of the batch processing, problem detection, integration and data management tasks required by the platform. The JobEngine (or more correctly JobEngines) execute a range of different tasks simultaneously and perpetually which are defined and configured via the Job Configuration form which is accessed via the Framework applications at the top right of the application screen as shown below:
Not all jobs are relevant to all customers, and many jobs are highly configurable. Merlot recommends that customers seeking to update or make changes to the configuration of any jobs, or to activate new jobs, consult with merlot Support in the first instance. Changes to job configuration may impact downstream systems or impact the way in which important processes are executed. Advanced technical information relating to the JobEngine and it's components can be found in the /wiki/spaces/DOC/pages/42008961section of this wiki. Please reach out to your account manager if you do not have access to this content.
The Job Configuration form, shown below, summarizes the data in these tables and presents it within the merlot.aero user interface, providing customers with information on job status, detailed log information as well as job configuration parameters.
The different columns in the primary table of the Job Configuration form are explained below:
Column Name | Description |
---|---|
Name | Provides an easily understandable description of the configured job instance. |
Type | Defines the job type. This field value should never be edited. |
ScheduleType | Defines how the the job instance is executed by the JobEngine. Not Scheduled (0) – The job is not scheduled and will not run unless an instance is queued manually via a function in the user interface, or manually via a script. Many jobs are "Active" but "Not Scheduled". Standard Timer (1) – The will run on a timer using the LastRun value and the RunEveryMinutes value. Restricted Timer (2) – The job will run on a schedule using "RunEveryMinutes" but not between the hours of "ScheduleStart" and "ScheduleEnd" which is configured separately as a job parameter. Daily Timer (3) – The job will run on a schedule at the same time each day between "ScheduleStart" and "ScheduleEnd" which is configured separately as a job parameter. Continuous (4) – The job will be re-added to the queue as soon as the previous instance has finished processing. |
TimeOutMinutes | Defines the number of minutes the job will continue processing before it is aborted. |
RunEveryMinutes | Defines the number of minutes between job instance executions when the job is configured to run on a schedule type. |
Last Run | Defines the last time (in Standard time mode) the job was executed. |
LastSuccessfulRunStart | Defines the last time (in Standard time mode) that the job executed successfully. |
Active | Defines if the job is Active. Inactive jobs are hidden by default. Not all jobs are required to be active. |
Users can see more details for a specific job, including instance history, job configuration and subscription targets by expanding the row for a given job, as shown below.
Job Parameters, which are used to configure how a job behaves are shown in the Config tab. Parameters include things like connection end-points for integrations, credentials and behavioral settings.
The different columns in the Config sub-table are explained below:
Column Name | Description |
---|---|
Name | Defines the name of the JobParameter. This value should never be edited. |
Value | Defines the parameter value. |
Default Value | Depending on the job, this column defines the default behaviour of the parameter if the value field is blank. |
Comments | Defines any freetext comments associated with the parameter. |
The history of job queue instances, as well as related detailed log information can be viewed from the Details tab and associated Log hyperlink against each job queue instance record.
The different columns in the Details sub-table are explained below:
Column Name | Description |
---|---|
Run At | Shows the time the job queue instance was started. |
Status | Shows the status of the job queue instance. This is a numerical value which is defines as follows: 0 = Awaiting processing / not processed 1 = Processing 2 = Success 3 = Failure 4 = Not required 5 = Restart required 6 = Cancelling 7 = Cancelled 8 = Aborted, job parameters do not match defined server or database |
Status Text | Shows detailed information about status information about the job queue instance, including the assigned JobEngine role, current state and time taken for processing. |
Action | Provides access to a hyperlink for accessing detailed log information for the job queue instance. |
Detailed log information for a given job queue instance can be viewed in the Job Log form. Log information can provide information, including technical stack traces, on the current status of a job. The job log data is typically used to determine why a job instance might be failing to process.
Job Instance Structure
The JobEngine works by reading the configuration of a specific job from the dbo.Job and dbo.JobParameter tables, and adding individual instances into the JobQueue and JobQueueParameter tables, after which they are processed by individual JobEngine "roles" (there are typically many of these roles running at any given time to achieve maximum performance via load balancing). For integration jobs, such as MVT message exports, the JobEngine will also reference the Subscription Framework to determine where the generated content should be distributed.
Individual Job Overview
The following table provides a listing of the individual jobs classes which can be executed by the JobEngine. They have been broken down into functional areas. It is important to note when reviewing this information that there may be multiple instances of the same job class configured in your merlot.aero installation, but with differing parameters. For example, you may have a number of job instances for the same class that are configured to process different date ranges on different schedules.
Under Construction
We are in the process of updating the Individual Job Listing section below. Updates to these page will be added soon!
Individual Job Listing
The following table provides information on each job within the merlot Job library. Detailed information on specific JobTypes can be accessed by clicking on the JobType value in the table (where available).
JobId | Group | Name | Type | Description | User Configurable | Default State | Self-hosted | Priority |
2 | Integrations | Import Sector and Delay Information | Imports flight data from an XML flat file stored in a UNC path. | Yes | Inactive | P2 | ||
25 | Integrations | ASM Message Export | Generates IATA ASM messages and populates the MessageOutbound table. | Yes | Not Scheduled | P1 | ||
28 | Integrations | Navtech CREWINFO Message Export | Generates and exports XML flat files containing crew information for consumption by Navtech Flight Planning applications. | Yes | Not Scheduled | P1 | ||
29 | Integrations | Navtech CREWLIST Message Export | Generates and exports XML flat files containing crew list data for consumption by Navtech Flight Planning applications. | Yes | Not Scheduled | P1 | ||
30 | Integrations | UNEDIFACT FCM Message Export | Generates and transmits UNEDIFACT FCM messages types to a target endpoint. Uses Subscription framework. | Yes | Not Scheduled | P0 | ||
36 | Integrations | ASM Message Import | Imports IATA ASM message data from the MessageInbound table. | Yes | Not Scheduled | P1 | ||
78 | Integrations | ASM Message Import | Imports IATA ASM message data from the MessageInbound table. | Yes | Not Scheduled | P1 | ||
39 | Integrations | Daniels IAS Export | Generates a defined flat file containing aircraft data for consumption by the Daniel's IAS engineering system. | No | UI Triggered | P1 | ||
41 | Integrations | Reservations Market Pair Import | Retrieves passenger data by market pair from the VRS reservations system. | Yes | Not Scheduled | P1 | ||
42 | Integrations | Flight Passenger Manifest Import | Retrieves passenger manifest data from the VRS reservations system. | Yes | Not Scheduled | P2 | ||
46 | Integrations | OOOI Times from SkyNet Import | ImportOOOITimesJob | Depreciated for mainstream customers. | N/A | N/A | P1 | |
49 | Integrations | Reservations Provider Pax Number Import | Retrieves Passenger counts from VRS reservations provider. | Yes | Not Scheduled | P1 | ||
53 | Integrations | TAA Payroll Export | Exports crew and schedule data for consumption by an external payroll system. Only used by Thailand AirAsia | Yes | Not Scheduled | P1 | ||
54 | Integrations | SkyNet Flight Position Import | Imports XML flight and tail position data from the SkyNet web service API. | Yes | Not Scheduled | P1 | ||
59 | Integrations | SSM Message Import | Imports IATA SSM message data from the MessageInbound table. | Yes | Not Scheduled | P1 | ||
60 | Integrations | SSIM Message Export | Exports IATA SSIM flat files to a UNC path. | No | UI Triggered | P1 | ||
65 | Integrations | NewSkies Reservation Data Import | Imports passenger and baggage count data from the Navitaire NewSkies reservations system. | Yes | Not Scheduled | P1 | ||
66 | Integrations | SkyNet Current Flight Import | Imports XML flight data from the SkyNet web service API. | Yes | Not Scheduled | P1 | ||
67 | Integrations | Navtech Flight Plan Import | Imports Navtech Flight Plan documents from an FTP location and imports them into the FlightFollowDocument repository. | Yes | Not Scheduled | P0 | ||
68 | Integrations | FlightFocus Flight and Crew Export | Generates and transmits XML flight and crew data to the Flight Focus EFB web service API. | Yes | Not Scheduled | P1 | ||
75 | Integrations | MVT Message Import | Imports IATA MVT message data from the MessageInbound table. | Yes | Not Scheduled | P0 | ||
76 | Integrations | <Customized> Email Message Import | Imports messages (typically IATA formats) from a POP3 email account into the MessageInbound table. | Yes | Not Scheduled | P2 | ||
80 | Integrations | Create Outbound Messages | Executes a SQL query and outputs the result set to the MessageOutbound table. Typically this is paired with a MessageOutbound export job which can transmit comma-delimited files to UNC/FTP/email. | No | Not Scheduled | P0 | ||
82 | Integrations | Sabre Crew File Export | Exports schedule and crew data a flat file for consumption by the Sabre CrewTrac application. | Yes | Not Scheduled | P1 | ||
84 | Integrations | MessageOutbound Export | Exports data from the MessageOutbound table to an external endpoint, e.g. FTP, UNC, BLOB, email. | Yes | Not Scheduled | P0 | ||
85 | Integrations | UNEDIFACT MCL Message Export | Generates and exports APIS EDIFACT MCL messages types to a target endpoint. Uses Subscription framework. | Yes | Not Scheduled | P0 | ||
87 | Integrations | FTP File Import | Imports data contained in flat files on an external FTP server into the MessageInbound table. | Yes | Not Scheduled | P0 | ||
88 | Integrations | SSM message Import Job. | Imports IATA SSM message data from the MessageInbound table | Yes | Not Scheduled | P1 | ||
89 | Integrations | Report on Table Change Export | InsertUpdateDetectionJob | Looks for new/updated records in a target table, and generates a report/message to be sent via email. | No | Not Scheduled | P2 | |
90 | Integrations | FlightMan Crew List Export | Generates and transmits XML containing crew assignment data to the FlightMan ICAS platform web service API. | Yes | Not Scheduled | P1 | ||
91 | Integrations | FlightMan Flight and Crew Schedule Export | Generates and transmits XML containing flight and crew schedule data to the FlightMan ICAS platform via web service API. | Yes | Not Scheduled | P1 | ||
94 | Integrations | Merlot Allowance Export Job | Exports payroll and allowance data from EmployeePairingAllowance to PeopleSoft web service API | Yes | Not Scheduled | P1 | ||
95 | Integrations | PeopleSoft Duty Assignment Export | Generates and exports crew duty assignment data to the PeopleSoft web service API. | Yes | Not Scheduled | P1 | ||
102 | Integrations | MVT Message Export | Generates IATA MVT message data into the MessageOutbound table. | Yes | Not Scheduled | P1 | ||
103 | Integrations | ACARS M22 Message Import | Imports M22 formatted ACARS data from the MessageInbound table. | Yes | Not Scheduled | P1 | ||
105 | Integrations | Jeppesen Schedule Export | Generates and exports XML flat files containing flight schedule data for consumption by Jepessen Flight Planning applications. | Yes | Not Scheduled | P0 | ||
106 | Integrations | Jeppesen Crew Export | Generates XML file containing crew information for consumption by Jepessen Flight Planning | Yes | Not Scheduled | P0 | ||
108 | Integrations | Jeppesen Flight Plan Import | Imports Jeppesen flight plan documents from an FTP location and imports them into the FlightFollowDocument repository. | Yes | Not Scheduled | P0 | ||
109 | Integrations | ADP Payroll Export | Generates and transmits a flat file containing payroll data to an FTP endpoint for consumtpion by ADP Payroll applications when triggered from CrewPayroll. | No | UI Triggered | P1 | ||
110 | Integrations | ACARS DEP/ARR Message Import | Imports DEP/ARR formatted ACARS data from the MessageInbound table. | Yes | Not Scheduled | P0 | ||
111 | Integrations | ACARS A80 Message Import | Imports A80 formatted ACARS data from the MessageInbound table. | Yes | Not Scheduled | P0 | ||
115 | Integrations | AMOS Future Flights Export | Generates and transmits XML file files containing future flight data for consumption by the AMOS engineering system. | Yes | Not Scheduled | P0 | ||
116 | Integrations | AMOS Executed Flights Export | Generates and transmits XML flat files containing completed flight data for consumption by the AMOS engineering system. | Yes | Not Scheduled | P0 | ||
117 | Integrations | Pelesys Qualification Import | Imports XML qualification data from the Pelesys LMS system via the Pelesys Galaxy web service API. | Yes | Not Scheduled | P2 | ||
1 | Messaging | Send Email Messages | Transmits email messages generated by merlot or initiated by a merlot user via an SMTP gateway. | Yes | Scheduled | P1 | ||
3 | Messaging | Send User and Employee Messages | Transmits internal merlot.aero messages between merlot users and employees. | No | Scheduled | P1 | ||
56 | Messaging | Send CrewMobile Notifications | Sends push notifications to the CrewMobile application. | No | Scheduled | P1 | ||
64 | Messaging | Send SMS Messages | Sends SMS messages generated by merlot or initiated by a merlot user via merlot's Bulletin SMS web service API gateway. | Yes | Not Scheduled | P2 | ||
8-10 | Problem Detection | Update Flight and Duty Times <Customized> | Updates pairing report/release times and calculated totals data based on flight timing changes. | Yes | Scheduled | P0 | ||
13 | Problem Detection | Check Crew Expiries | Detects issues with approaching course expiries and populates problems in the Monitor panels. | Yes | Scheduled | P1 | ||
14 | Problem Detection | Run Rules on Change Pairings and Duties <Customized> | Processes changed pairings against the RulesEngine and populates violations in the Monitor panels. | Yes | Scheduled | P0 | ||
32 | Problem Detection | Run Rules for Roster Validation - Adhoc | Processes pairings against the RulesEngine and populates violations in the CrewRoster Monitor panels for validation of rosters in the planning state. | Yes | UI Triggered | P0 | ||
62 | Problem Detection | Re-valuate Rules for Outstanding Violations | Processes pairings which has outstanding violations in the Monitor panels. | Yes | Scheduled | P0 | ||
63 | Problem Detection | Re-valuate Rules for Existing Expiry Violations | Processes pairings with outstanding course and certification expiry violations when expiries are updated via CrewPlan. | Yes | Scheduled | P0 | ||
15 | Problem Detection | Check Static Data | Generates problems in the Monitor panel if there are static data issues e.g. ports without a valid time zone. | No | Scheduled | P1 | ||
16 | Problem Detection | Check Crew Port Currency | Detects potential port currency issues and populates problems in the Monitor panels. | Yes | Not Scheduled | P1 | ||
17 | Problem Detection | Check Crew Route Currency | Detects potential route currency issues and populates problems in the Monitor panels. | Yes | Not Scheduled | P1 | ||
18 | Problem Detection | Check Duty Event Order | Detects and repairs issues with the logical order of PairingActivity records. | No | Scheduled | P1 | ||
21 | Problem Detection | Check Crew Certification Expiries | Detects issues with approaching certification expiries and populates problems in the Monitor panels. | Yes | Scheduled | P1 | ||
31 | Problem Detection | Check Instructor Assignment | Detects missing instructors from course instances and populates problems in the Monitor panels. | Yes | Not Scheduled | P1 | ||
33 | Problem Detection | Evaluate Seat Qualification Date | Generates a seat qualification value date/time value in the Employee.QualifiedUntil column which is displayed and used by CrewPortal. | Yes | Not Scheduled | P1 | ||
81 | Problem Detection | Check Flight and Pairing Complement | Detects issues with flight and pairing complements and populates problems in the Monitor panel | Yes | Scheduled | P1 | ||
6 | Reports | Generate Roster Reports for CrewPortal | Generates report/links for the Employee Roster Report which are made available in CrewPortal. | Yes | Triggered by UI | P1 | ||
12 | Reports | Generate Briefing Reports for CrewPortal | Generates report/links for the Employee Briefing Report which are made available in CrewPortal. | Yes | Scheduled | P0 | ||
22 | Reports | Generate General Declaration Reports for Portals | Generates report/links for the General Declaration Report which are made available in CrewPortal and AircraftPortal. | Yes | Not Scheduled | P0 | ||
45 | Reports | Generate Voyage Reports for CrewPortal | Generates report/links for the Voyage Report which are made available in CrewPortal and AircraftPortal. | Yes | Not Scheduled | P0 | ||
1022 | Reports | Generate Payroll Summary Report for CrewPortal | Generates report/links for the Payroll Summary Report which are made available in CrewPortal. | No | UI Triggered | P2 | ||
4 | System | Update Crew Portal Duty Changes | Generates duty change notifications within CrewPortal. | No | Scheduled | P0 | ||
5 | System | Update Crew Portal Sign On | Generates summarized sign on information for use by CrewPortal and CrewMobile. | No | Scheduled | P0 | ||
11 | System | Update Crew Portal Pairings/Duties | Generates summarized pairing information for use by CrewPortal and CrewMobile | No | Scheduled | P0 | ||
20 | System | Run Ad-hoc SQL <Customized> | Executes a SQL command script updating, inserting or deleting data. This job type does not return a row set. | Yes | Adhoc/Scheduled | P1 | ||
47 | System | Delete Aged Job Log Records | Executes a SQL command to delete old records from the JobLog tables. | No | Scheduled | P1 | ||
48 | System | Check Database Size | Executes a SQL command and emails support once a database reaches a certain size. | No | Scheduled | On Request | P1 | |
79 | System | Delete Old Audit Records | Executes a SQL command to delete old Audit table entries. Not required when running in Archive mode. | No | Not Scheduled | On Request | P2 | |
23 | System | Deploy DAC Pack | DeployDacPackJob | Deploys a DACPAC against the target database. | No | Adhoc | Not Applicable | P2 |
24 | System | Update Employee, Equipment and Port Restrictions Matrix | Maintains the restriction framework matrix based on changes to various entities. | No | Scheduled | P1 | ||
27 | Problem Detection | Check OnwardFlight Data | Monitors and repairs incorrect onward flight information | No | Scheduled | P1 | ||
34 | System | Populate warehouse.* Schema Tables | Performs ETL of flight data for population of the data warehouse tables. | No | Scheduled | P1 | ||
35 | System | Update PortTimeOffset Table | Detects issues with port information when any port-related entities are updated relative to the PortTimeOffset table. | No | Scheduled | P1 | ||
37 | System | Update Linked Qualifications | Updates the linked qualification elements defined against a course in CrewPlan when employees are added or deleted from a course instance. | No | Scheduled | P1 | ||
38 | System | Allocate Employee Requests | Allocates employee leave requests entered from CrewPortal when is triggered from the user interface. | No | Not Scheduled | P1 | ||
40 | System | Update Employee Points Calculation | Maintains points scores used in the leave request allocation process. | No | Not Scheduled | P1 | ||
43 | System | CPSwapCheckOffer | CPSwapCheckOfferJob | Depreciated for mainstream customers | N/A | N/A | P0 | |
44 | System | CPSwapPerformSwap | CPSwapPerformSwapJob | Depreciated for mainstream customers | N/A | N/A | P0 | |
50 | System | Deploy Reports | DeployReportsJob | Deploys an updated report package to the target server. Deprecated for mainstream customers. | No | Adhoc | P2 | |
51 | System | Run Pay / Allowance Rules | Processes allowance and payroll when triggered from the CrewPayroll user interface. | No | UI Triggered | P1 | ||
52 | System | Calculate Flight Performance Data | Performs ETL of flight data for population of the data warehouse tables. | No | Scheduled | P2 | ||
55 | System | Export Optimisation XML - Ad-hoc only | Generates XML files containing pairing, crew, bid and pre-allocation data for consumption by merlot optimization partners. | Yes | UI Triggered | P1 | ||
57 | System | Delete Unsubscribed Mobile Devices | Removes unauthenticated mobile device records for CrewMobile and AircraftMobile from the database | No | Scheduled | P2 | ||
58 | System | Rebuild Table Indexes | Rebuilds table indexes which meet certain fragmentation thresholds. | No | Scheduled | On Request | P1 | |
61 | System | Backup Database to Azure Blob Storage | Performs a backup of the OLTP database to an Azure BLOB storage endpoint. | No | Scheduled | Not Applicable | P1 | |
69 | System | Evaluate Crew Swaps | Evaluates and processes crew swaps (trip trades) submitted via CrewPortal. | Yes | Not Scheduled | P0 | ||
70 | System | Create Employee Roster Periods | Monitors for missing or incorrect employee roster period records and updates as required. | No | Scheduled | P1 | ||
71 | System | Resize Employee Images | Monitors and actively resizes employee profile images stored in the database. | No | Scheduled | P1 | ||
72 | System | Roster Solution Import | Imports solutions from a merlot partner roster optimizers. | No | Triggered by UI | P1 | ||
73 | System | Generate Rostered Pairing Versions | Generates a snapshot of pairing data at the time of roster publish. | No | Triggered by UI | P1 | ||
74 | System | Archive Audit Data | Archives data from the Audit table in the OLTP database to an archive database. | Yes | Scheduled | On Request | P1 | |
83 | System | Generate FlightFollowPerformance Data | Performs ETL of flight data for population of performance data | No | Scheduled | P1 | ||
86 | System | Update FlightFollowPerformance table | UpdateFlightFollowPerformanceJob | Performs ETL of flight data for population of performance data | No | Scheduled | P1 | |
92 | System | Update Usage Statistics | Generates usage telemetry for cloud-based customers. | No | Scheduled | Not Applicable | P1 | |
93 | System | Update IIS Log Usage Statistics | Generates usage telemetry for cloud-based customers. | No | Scheduled | Not Applicable | P1 | |
96 | System | Update CPHistories | UpdateCPHistoriesJob | Depreciated for mainstream customers. | N/A | N/A | P2 | |
97 | System | Update CPRecencies | UpdateCPRecenciesJob | Depreciated for mainstream customers. | N/A | N/A | P2 | |
98 | System | Import Combined DST and Locations | Imports combined DST and locations from text file and saves to the database. | No | Adhoc | Not Applicable | P1 | |
99 | System | Generate Open Time Swaps | CreateUpdateCPSwapsJob | Depreciated for mainstream customers. | N/A | N/A | P2 | |
100 | System | Generate Open-Time Swap Data | Generates open time pairing information for display within CrewPortal. | Yes | Not Scheduled | P1 | ||
101 | System | Run Pairing Optimiser | Optimiser.RunPairingOptimiserJob | Initializes and monitors a pairing optimization run from the CrewSolve application. | No | UI Triggered | P1 | |
10000 | System | Update Flight Log Recency Assumption Data | Generates flying recency records for customers employing an assumptive model for recency rules (i.e. where actual flight log data is not entered manually or imported from ACARS message data). | Yes | Not Scheduled | P1 | ||
Create a PDF version of a report and send it via email | ||||||||
Generate custom CREWINFO messages | Generate custom CREWINFO messages | |||||||
SAFE FRMS roster export job | SAFE FRMS roster export job | |||||||
Original roster converter job | ||||||||
Route Cosmic Radiation Dosage Import Job - On-Demand | ||||||||
Export Optimiser Solution - AdHoc | ||||||||
Run Routing Optimiser - AdHoc |