Agent Repository is a central place where agents are stored and maintained that allow developers and other users to commit and retrieve agents, collaborate on different projects, etc.
Agent Repository has a folder-like structure to make it easier to logically organize your agents.
Another important feature of Agent Repository is that it acts as a version control management system, allowing you to keeps all the versions of your agents, with various options available to manage and monitor your agents. You can check-in your agent in the repository and can keep a track of your versions and changes in the agent. The version control system records the changes you make to your agents, so it lets you save a snapshot of your agents at any time you want.
Along with version management, there are several other useful options like scheduling an agent, deploying an agent, executing an agent, jobs and run management, server and cluster deployments which makes it a one-stop solution for managing and monitoring your agents from anywhere you want.
ACC Web Portal
Agent Repository in the ACC Web Portal offers a feature-rich web-based interface which allows you to view, organize and maintain your agents, as well as deploy, schedule, run and monitor them via a web browser.
Navigate to Agents -> Repository in order to access Agent Repository.
There is an option to filter agents by selecting a cluster/server from the drop-down list, or you can search agent in the repository by typing an agent name into the 'Search agents' text box.
Then, you should be able to see a list of all agents, directories, shared folders and configs.
A general feature of sorting, which allows sorting on the basis of alphanumerical values and date values.
1. Name: Sort by Alphanumerical order.
2. Rate Limit: Sort by the smallest or largest number.
3. Last updated: Sort by the latest or the oldest dates.
4. Tickets: Sort by the number of tickets.
Please note that the Agent Repository page does not refresh automatically in a real-time and some information about agents, e.g. Job Status, Run Status, etc. may change often. So, you should consider using the 'Refresh' button to refresh the Agent Repository page and get the latest snapshot.
Authorization to directories and any objects inside of these directories can be controlled with Privileges. Privileges control if a user can modify a directory or an object inside of that directory.
A privilege is the right to perform a particular action or to perform an action on an object of a particular type. Objects include Agents, Config files, Shared Folders, etc. Actions include Full Control, Read Only, Delete, etc.
Privileges also control a user’s access to a ticketing system, whether or not a user receives email notifications, etc. For example, a user can receive an email notification when an agent ran successfully or when it failed.
Only Admins and Super Admins can grant or revoke privileges.
As mentioned earlier, Agent Repository has a folder-like structure to make it easier to logically organize your agents. Folders are called Directories in Agent Repository. You can create as many directories and sub-directories as you wish to organize your agents in a way that is meaningful for and searchable by the entire team.
A number of actions can be performed on a current directory and sub-directories, including creating new sub-directory, edit, delete, setting directory privileges, creating a new config, etc.
Click the 'Directory' action button at the top of the agent list or click the action button next to a sub-directory to access the context menu.
- Edit Directory or Edit: Edit a current directory or sub-directory settings.
- Directory Privileges or Privileges: Edit a current directory or sub-directory privileges.
- New Subdirectory: Create a new sub-directory inside of a current directory.
- New Config: Create a new config.
- New Schema: Create a new schema.
- Directory Audit Trail: Demonstrate the sequence of activities or operations performed within the directory or sub-directory by the user.
- Delete: Delete a directory and ALL agents it contains.
- Audit logs: Demonstrate the sequence of activities or operations performed within the selected directory or sub-directory by the user.
Create New Sub-Directory
To create a new sub-directory inside of a current directory, use a New Subdirectory action command. In a pop-up screen enter a Directory name which is a mandatory field.
The rest of the fields and checkboxes on this screen are optional.
- Default support user: Set a default support user for this directory by selecting from the drop-down list. If you set a support user, then all failure tickets will automatically assign to the default support user. Support users are those who analyze and respond to the tickets.
- Default consumer user: Set a default consumer user for this directory by selecting from the drop-down list. Consumer users are users for which tickets are created to report the issues.
- Run failure tickets: If you check the “Run failure tickets” checkbox, then it will automatically create a ticket when any of the agent runs in this directory get failed.
- Job failure tickets: If you check the “Job failure tickets” checkbox, then it will automatically create a ticket when any of the agent jobs in this directory gets failed.
- Log agent and API activity: If you check the “Log agent and API activity” checkbox, then it will create log of agent and API activity.
- Environment: By default "Prod" is set as a production environment.
- Default log level: By default log level is set as "High" but a user can also change the log level to None, Low or Medium.
- Run history retention: A user can specify retention days to run the history of agents, else it will take the default value of retention.
- Log retention: A user can specify retention days to log of agents, else it will take the default value of retention.
- Ticket retention: A user can specify retention days to log of Tickets, else it will take the default value of retention.
- High CPU Action: This demonstrates the action to be taken when the CPU usage is High of the running agent. A user can set this with these options- Default, None, Raise Ticket, Stop Agent, Stop Session.
To edit a current directory or sub-directory settings, use a Edit Directory or Edit action command. The Edit Directory pop-up screen looks exactly the same as the Create New Directory pop-up screen. Please read Create New Sub-Directory section for more details.
To set and edit directory privileges use Directory Privileges or Privileges action command.
In the pop-up screen select a user or role from a drop down list.
Once selected, you will see a list of Privileges as shown below. Select the privileges you wish to assign to a selected user or role.
To check complete details about an agent, use context menu associated with each agent row on Agent repository page and select "Agent Details". This will display complete details about the agent, such as latest check-in description, proxy used, success criteria used in the agent for deciding the execution status, export targets like CSV/Excel/Parquet etc., data distribution such as AWS S3, Azure, FTP/sFTP, etc. and Data Validation rules applied on the agent such as empty row handling, duplication row handling, empty table handling and validation error handling which decides to remove or fail the agent basis the data type applied on the agent and also the allow/disallow empty data for an export column.
Click the Agent Details link to see complete details of the agent.
When you click on the Agent Details link, it will open a new page with all the agent details mentioned above.
There is a number of different actions that can be performed on agents by selecting an appropriate option from the context menu. Click on the action icon located at the right-hand side of each row of the Agent Repository to expand the context menu. Various menu options give you the flexibility to run, manage and monitor your agents and their status.
Sample Context Menu
The availability of different context menu items may vary depending on your privileges, the status of the agent, whether the agent has been deployed or not, whether the agent has been deployed to a cluster, etc.
Run on Cluster
This command allows you run your agent on a cluster once it has been deployed to a cluster. This menu option will not be available if an agent has not been deployed to a cluster yet.
Run on Cluster
You can see a cluster name an agent has been deployed to and an agent version number.
This command allow users to configure Job settings of the agent. Please refer to Jobs for more information.
A user can see all schedules set for the agent and also do modification such as adding schedules, edit, delete etc. Please refer to Jobs for more information.
This command allows the user to move the agent from the current directory to another one.
Click the Move Agent option and it will open a new pop-up form giving you an option to enter and select a name of the directory you want to move your agent to.
A user can use this command to add inputs files in the agent. It can be either done by Upload file or Create CSV File option. These input files can also be modified with Edit Content command in the Context menu associated with each row, also a user can disabled, download or deleted the input files.
Please note that a user can only upload maximum 100 MB size of input file at a time. However, multiple input files can be added.
Often, it is useful to know who did what and when to a particular agent. Audit Logs is kept for all the agents in Agent Repository. Audit Logs is a chronological record of system activities to enable the reconstruction and examination of the sequence of events and/or changes in an agent.
For example, you will be able to see who and when created and modified an agent, who and when deployed it, who and when added a job or schedule, etc.
Use the 'Audit Logs' command from the context menu to open the Agent Audit Trail page.
On the above screen you will see the actions, changes and event have been performed on an agent. Additionally, when you click on the Raw Data link for any audit row, it will open a pop-up screen which will show you the Audit Log Content for that particular audit row. These log details are stored in database.
On the other hand, click on the “File Logs” link displayed on the top right side of "Agent Audit Trail" page to check logs stored in the file system.
To know more information about agent audit logs, please read Audit Trail article.
The Delete command allows the user to delete agents from the repository, including all the associated deployments, schedules, execution histories, etc. So, you need to be very cautious before using the Delete command because you will not be able to undo this operation. It is recommended that take a back up of your agent before deleting it from the repository. An alert frame is still popped-up before doing the permanent delete.
This operation will delete an agent, including all the associated deployments, schedules, execution histories, etc. You will not be able to undo this operation.
A user can deploy agents to a cluster. Please note that you need to set up your servers or/and clusters before. Then only be able to deploy your agents. Check Servers & Clusters for more information about servers and clusters, how to set them up and differences between them.
Deploy to Cluster
This command allows you to deploy your agent to a cluster. This menu option will be available in the Latest Version of the agent.
Deploy to Cluster
Read Cluster Deployment or more information on how to deploy agents.
Deploy Latest Version to Cluster
A user can deploy the latest version of the agent available in agent repository but yet not deployed to the cluster by using same command "Deploy to Cluster". You can see an agent version number you are going to deploy.
Read Cluster Deployment for more information on how to deploy agents.
A user can see all the checked in versions of an agent with Version History option. The page displays the previously checked in versions of the agent, description of changes, Sequentum Enterprise version, a date when each version was added into the agent repository, its deployment status, and the username who added that version.
Deploy Older Version to Cluster
If there is any requirement where an older version of the agent needs to be deployed then the same can also be achieved.
Agents -> Repository -> Version History
Version History page displays the existing versions of the agent, Version description, Sequentum Enterprise Version, "Added date and time" and User.
Click on the context Menu of the selected version for menu details:
Deploy to Cluster - Click on this option to deploy selected version on the cluster.
Download Version - A user can download the agent(scgx) of the selected version on their local machine. Storing this version will override the existing version.
Delete - A user can deletes a version from the agent repository with this option. A version can only be deleted if there is no current reference to it. (i.e that version of that agent is not currently deployed on any cluster). Please note that a User with Admin privilege can only be able to delete agent versions.
Remove from Cluster
This command allows you to remove an agent from a cluster. For more information please read Cluster Deployments.
Remove from cluster
This command allows you to change agent deployment from one cluster to another.
For more information please read Cluster Deployments.
This command allows you to view Run History of the agent. Run history page displays the complete run history of agent job schedule.
Agent Run History
You can navigate to the Run history page via below option:
- Agents→Schedules→Select Server or Cluster→Agent Context Menu→Run History
By clicking the 'Run History' you should see Run History of the selected Agent.
The agent Run History report contains the following columns:
RunId: This column shows an increasing number for each row w.r.t the agent run.
Session: This column shows sessions of agent run.
IP: This column shows server IP associated with the cluster on which agent executes.
Start/Queue Time: This column displays the date and time at which the run started executing on the cluster.
End Time: This column displays the date and time at which the run finished its execution on the cluster.
Run Time: This column displays the total execution time of the run.
Actions: The successfully executed action commands.
Pages: Actual page loads. This is all pages that are loaded into the main frame of the browser or pages loaded by the HTML/JSON/XML parser.
Requests: All requests made to the target web server, including HTML, images, styles, script, etc. For HTML/JSON/XML parsers there's one request per page load.
Errors: All action errors. Note that a page load error that's retired and then successful does not result in an error, but the error counts as a page load if a page was actually loaded.
Data: The number of records that have been extracted
Delivery: Actual main data rows delivered to the destination, such as
Status Message: This column describes the agent run status.
Status: This column shows the run status as Failure or Succeeded.
Last Updated: This column shows day or time of the last executed run.
Run History Context Menu
For each row, under the run history page there are several context menu options that can be used to manage the agent execution, check logs, download export data and manage agent run status.
- Get Log: Click on this option and it will open an new window with log contents of the agent run.
- Get Data: Click on this option and it will download your export data in a compressed zip format allowing you again to view the output data without having to go to the server for checking the export data.
- View Input Parameter: Click on this option to view the parameter named values that get their value assignments when an agent starts.
- Restart: Click on this option if you want to restart your agent.
- Retry Errors & Continue: This is a useful option in case your agent encountered any failures and you want to only retry the failures/errors and don't want to start the restart again. Click on this option and the agent will retry the errors and will continue the agent execution for the failures.
- Set Failed Status: Use this option to set the run status to failed which allows the users to stop the agent execution.
- Set Success Status: Use this option to set the run status to success if it failed for the last execution and the next run execution is getting blocked due to the failed run status.
- Reset Success Criteria: Click this option to clear data of success criteria so that next run gets initiated with a fresh start.
- Delete: Click on this option to delete run entry from the run history page.
Once your agent is successfully deployed to a cluster, you can schedule it to run automatically on a regular basis.
Please read the following article for more details:
Click on Agents -> Running Agents tab to see all the running agents for an Organization. A user with Admin privilege are only allowed to see the running agents in an organization. However, Super Admin privilege can see the running agents across the organizations.
Rate limits can be used to control how hard agents are allowed to hit a target website. For example, a rate limit can be added for a certain website and be configured to allow a maximum number of page loads over a given time period. The rate limit is then assigned to all agents targeting this website, and the agents will stop once the maximum number of allowed page loads is reached.
A rate limit can be assigned to an agent by clicking on the check or cross icon in the Rate Limit column in the agent repository.
Please read the following article for more details:
Sequentum Enterprise Editor
Agent Repository in the Sequentum Enterprise Editor offers a feature-rich web-based interface which allows you to view, organize and maintain your agents, as well as deploy, schedule, run and monitor them via a web browser.
To access Agent Repository click the cloud icon in the right top corner of the Sequentum Enterprise and select Agent Repository command from the context menu. You must be logged in, otherwise you will not see the cloud icon and will not be able to access Agent Repository.
Accessing Agent Repository in Sequentum Enterprise
Then, you will see Agent Repository in a new window which contains a list of all the Directories, Agents, Shared Folders, Configs, etc.
You can navigate through Agent Repository like you would through directories and files in a standard file system.
Create New Directory
To create a new directory in Agent Repository, click the 'New' button at the top right side of the window.
A pop-up window will display starting with a root directory named as Sequentum in which you can create sub-directories.
Enter a directory name and click the Ok button.
There is a number of different actions that can be performed on agents by selecting an appropriate option from the multi-level context menu. Right click on an agent to expand the context menu. Various menu options give you the flexibility to deploy, run, manage and monitor your agents and their status.
The availability of different context menu items may vary depending on your privileges, the status of the agent, whether the agent has been deployed or not, whether the agent has been deployed to a single server or a cluster, etc.
One of the important features of Agent Repository is that it acts as a version control management system, allowing you to keeps all the versions of your agents, with various options available to manage and monitor your agents. You can check-in your agent in the repository and can keep a track of your versions and changes in the agent. The version control system records the changes you make to your agents, so it lets you save a snapshot of your agents at any time you want.
In order to store an agent in the Agent Repository, right click on the agent, hover down to the cloud icon called 'Version Control', and a down drop menu will appear which allows you you perform the following actions: check in agent, view version history, get latest version, compare with latest version.
Version Control menu
Check In Agent
You should use Check In Agent command to add a new agent or a new version of an agent to the Agent Repository.
There is a couple of different way of checking in an agent.
The first way is to right click on the agent, hover down to the cloud icon called 'Version Control', and then select 'Check In Agent' command from a down drop menu. The second way to check in an agent is trough the main menu by navigating File->Check-In Agent.
Check In Agent Options
A new 'Check In Agent' pop-up window will open which allows you select an agent repository directory, add an agent/version description, and a number of other settings.
Check In Agent pop up window
You will see the following fields and settings in the 'Check In Agent' pop-up window:
Repository - When checking in an agent to the agent repository, a repository directory must be specified as to where to store the agent.
Version - Version shows the latest version of the agent in the repository and the current version of the agent that is being checked in. The standard practice is to increase the least significant number if making small changes and the more significant numbers when making larger changes.
Retention - Retention specifies how many records will be shown in the version history. By default, this field is set to 10.
Agent / Version Description - The version description and list of changes and bug fixes can be specified here.
Include Input Data - Input data are additional files that are provided in the agent folder. If the agent is using a data list to iterate through a CSV/Excel file, then this field needs to be checked so that the agent can access the input agent when it is checked out of the agent repository. By default, this field is checked.
Include Agent Schedule - This option will include the agent schedule in the version of the agent. If this option is checked, and then the agent is deployed, the schedule will be on the server in which the agent was deployed. By default this field is unchecked.
Deploy - Deploying an agent will automatically deploy the latest version of the agent to the server which is configured as the default deployment. By default this field is unchecked.
Production Ready - This option specifies if this version of the agent is ready for production. Generally, it is advised to deploy production-ready versions to a single server or a cluster. When checking in versions with small fixes, it might be better to not check in the agent in production mode and instead test the versions first before deploying. By default this option is unchecked.
Ignore Version Conflicts - When checking in an agent that is not built from the latest version, a small warning prompt will appear saying that there might be a version conflict when checking in this version.
Selecting this option will allow the user to check in an earlier version of an agent which overrides all the changes made in the latest version. For example, a user version is 1.0.5, the latest version in the repository is 1.0.6. User makes a small change and decides to check in version 1.0.7. If this option is selected and version 1.0.7 is checked into the agent repository, the changes made in 1.0.6 will not be reflected in 1.0.7. By default this option is unchecked.
View Versions / Version History
Version History shows the previously checked in versions of the agent up to the number of retention specified when checking in an agent. It includes a description of changes along with each version, a date when each version was checked into the agent repository, whether that version is ready for production, and the username of the individual who checked in that version.
Version History of the Agent
If you right click on any of the versions, you will see a context menu with the following commands:
Get Version - Saves the selected version of the agent locally. If another version of the agent already exists, the previous version will be overridden.
View Agent - View agent allows the user to get an overview of the agent, its fields, and how the agent is structured.
Compare with Local Version - This option compares the version that was selected with the current local version. The fields that are highlighted are the fields that have been changed.
Comparison Between Selected And Local Version
Delete - Deletes a version from the agent repository. A version can only be deleted if there is no current reference to it. (i.e that version of that agent is not currently deployed on any of the servers)
This process is permanent, and the version cannot be recovered if it is not saved locally.
Version 1.0.1 Has Been Deleted
Get Latest Version - Retrieves the latest checked-in version of the agent locally. If another version of the agent already exists on your local, then the previous version will be overridden.
You can restore the agent to your preferred location on your local and you can also restore the schedules if the agent was checked in with a schedule.
Compare with Latest Version - This option compares the latest version of the agent and the version currently available on your or the version currently open in Sequentum Enterprise.
The fields that are highlighted in red are the fields that have been changed.
To make modification in the directories, do right click on a specific directory and you can see a list of action commands like Rename, Move, Download, Deploy to Cluster and Delete.
Rename- If you want to change the directory name then click on Rename and update the name of the directory.
Move - You can move a directory from one location to another. Browse the path wherever you want to place the directory and click on the Move button.
Download - Download directory on your local system.
Deploy to Cluster - This will deploy all contents of the directory on Cluster.
Delete - It will delete the directory.
Add Shared Config
Configuration files are basically initialization files with a basic structure composed of sections, properties, and values. They are simple text files which contain key or property and every key has a name and a value.
Agent properties can be set with configuration file and used to apply on a single/current agent or multiple agents.
To apply configuration file on multiple agents, select the directory path where the agents are stored and click on Add Shared Config shown at the bottom left side of the agent repository window. The displayed window will show Name, Description, and Configuration. Add details and Save the file.
At the bottom left side of the config window, there are options to Validate and Apply to Current Agent. You can ratify the configuration by clicking on Validate. “Apply to current agent” will ask if you want to apply the configuration to all current agents placed in the present directory.
To share the config file with one or more servers, do right click on the config file. It will show a list of commands to edit, delete, download and deployments. Select Deployments and from the displayed window and click on Add Server or Cluster. Select server or cluster from the drop-down list and check on Deploy config to server. Click on Save and it will deploy the config file within the directory of that specific server.
The deployment can also be deleted by right-clicking on a specific server and then delete.
To deploy config file on each server, click on Deploy to All. It will apply to each and every agent placed in the directory of all servers.
Add Shared Folder
A shared folder is a shared resource that is made accessible to multiple users. Here in agent repository, files from your local system can be shared through the shared folder and these files are accessible by multiple agents.
To create a Shared Folder, locate a path in agent repository where shared folder needs to be created. Click on Add Shared Folder at the bottom left side of the displayed window. Browse the file from your local system and add a description of that file in the Description box which is optional. Click on Save to add the folder in the repository.
To deploy Shared Folder on one or more servers, right click on the folder and select Deployments.
Click on Add Server and select server from the drop-down list. Check to Deploy shared folder to server check box. Save it to deploy the folder within the specific location path on that server. The folder can be deployed on multiple servers as per the need.
To deploy the shared folder on all servers in one go, just click on Deploy to All Servers. It will share the folder on every server with the same location path as in the agent repository.
An audit log helps to monitor data and keep track of internal use of operations or we can say audit log is a chronological record or set of records that provide evidence of sequence of activities that have affected at any time specific operation, procedure, or event.
To Know the audit log of any specific agent, right click on agent and select Audit Log. It will display a window where you get to know the details of operation performed, its type, description, user who performed the operation and its timestamp.
An agent can be moved from one directory to another. Just select the specific agent and right click on it. It will display a command menu, select Move and browse the directory path where you need to move that agent. Click on Move and it will place the agent from current directory to the new directory path.
This menu option deletes the agent from the repository. By deleting agent from repository, it will delete all the associated deployments, schedules, and execution histories, so you need to be very cautious before using this option and you must take a back up of your agent somewhere before deleting. An alert frame is still popped-up before doing the permanent delete.
Refresh All Agents
To know the real-time update of the agents in the repository, select a directory where the agents are stored. Right click on any agent and select Refresh All Agents from the commands menu. It will refresh properties of all agents and we can see updated details in the repository.
An agent can directly be executed with this option. This option only works if the agent is deployed on the cluster.
This command will display a historical view of the agent run. Run history page shows the complete run history of agent job schedules.