Python Scripting
How to set up Python Environment in Sequentum Enterprise
NOTE:
Sequentum Enterprise currently supports Python 3.7 only and the same must be installed at C:\Program Files\Python37 path for all users on the Server or Desktop running with Sequentum Enterprise software.
For using Anaconda3 with Sequentum Enterprise, you should use the latest installer 2020.02 only, it works with the latest version of Sequentum Enterprise.
Python 3.7 recommended installation steps
Step 1: Download the Python 3.7 64-bit Windows Installer.
Step 2: Run the Installer.
Step 3: Check the Python version installed on your Windows machine using command prompt and install the popular Python Pandas library:
Open command promt and type python --version to get the python version installed.
Use pip command to install any external Python library
For example:
Use pip install pandas to install the Pyhton Pandas library on your machine.
Anaconda3 recommended installation steps
Step 1: Download the Anaconda installer.
Step 2: Run the Installer.
NOTE:
Install Anaconda to a directory path that does not contain spaces or Unicode characters.
To see if the conda installation of Python is in your PATH variable, open an Anaconda Prompt and run echo %PATH%.
To see which Python installation is currently set as the default, open an Anaconda Prompt and run where python.
To install the Pandas library in Anaconda, open an Anaconda Prompt and run:
conda install -c anaconda pandas
Once the Python3.7 or Anaconda3 is set-up on your machine, follow the below steps to start using Python with the Sequentum Enterprise software:
Go to Agent Command and click the edit option.
Go to the Properties tab.
In the Agent, Properties go to Scripting.
Go to Python Engine, which specifies the type of Python engine you want to use in your agent, and select Embedded which is set by default and uses IronPython or External which supports Python 3.7 CPython libraries.
Embedded: To use the IronPython which is an open-source implementation of the Python programming language which is tightly integrated with the .NET Framework, select the Embedded Python Engine option.
External: To use the full Python 3.7 which supports the CPython libraries such as Numpy and Pandas, select the External Python Engine option. Anaconda 3 is recommended as the python external engine. Both the External Python Engines must be installed separately on the Windows machine.
Go to Environment and select the environment variable for the agent such as None, Anaconda 3, Python37, Custom based on Python Engine property value. If you select the Embedded as a Python engine, then you need to remain with the default environment value i.e. None. If you select the External as Python engine, then the Environment property needs to be set to either Anaconda3 or Python 37 or Custom value depending upon the Python setup on the Windows machine.
Go to Custom Environment and specify the custom environment variables for the agent If you selected the Environment property as Custom in step 5. Individual environment variables must be separated by a new line. Example: PATH = C:\Program Files\Python37
Working With Python Intellisense in Sequentum Enterprise
Sequentum Enterprise uses third-party SyntaxEditor for the Python IntelliSense support which can be used for showing all the Public and All Methods supported by a particular Python library.
The following example shows the use of the external library Pandas in the Sequentum Enterprise Python Scripting editor and the use of the Python Intellisense with help of dot (.) to show all the Public and All the methods supported by the Pandas Library:
import clr
import pandas as pd
import os
from Sequentum.ContentGrabber.Api import *
def ProvideData(args):
filename = os.path.join(args.Agent.InputDirectoryName, "input-file.csv")
records = pd.read_csv(filename)
deduped = records.drop_duplicates(['Store ID','Variants'])
csv = deduped.to_csv(index=False, header=False)
data = csv.split('\n')
cds = CustomDataStorage()
cds.AddStringColumn('Store ID')
cds.AddStringColumn('Variants')
for x in data:
if x != '':
cds.AddRow('Store ID', x.split(',')[0], 'Variants', x.split(',')[1])
return cds
Pandas Intellisense
In the above example, you can see that we have used methods like args.Agent.InputDirectoryName or cds.AddStringColumn or cds.AddRow. These methods are Sequentum Enterprise C# methods which can be used by using from Sequentum.ContentGrabber.Api import * which enables you to use the Sequentum Enterprise inbuilt methods.
You can use the Sequentum C# Functions, Methods, and Script Utilities in Python scripting as well, by using the Python/DotNet context Intellisense support added for the C# methods in the Sequentum Enterprise Python Scripting editor.
The following example generates the complete path and filename placed in the input folder of the agent using C# and Python Scripting:
C#
PythonPython
using System;
using Sequentum.ContentGrabber.Api;
using System.IO;
using Sequentum.ContentGrabber.Commands;
public class Script
{
//See help for a definition of ContentTransformationArguments.
public static string TransformContent(ContentTransformationArguments args)
{
//Place your transformation code here.
//This example just returns the input data
var filename=args.GlobalData.GetString("filename");
var filePathWithfileName=Path.Combine(args.Agent.InputDirectoryName,filename);
return filePathWithfileName;
}
}
import clr import os from Sequentum.ContentGrabber.Api import * def TransformContent(args): filename=args.GlobalData.GetString("filename") filePathWithfileName=os.path.join(args.Agent.InputDirectoryName,filename) return filePathWithfileName
The Sequentum Enterprise C# methods like args.GlobalData.GetString,args.Agent.InputDirectoryName args.Agent, or args.DataRow etc. can also be used through the Intellisense support added to the Python Scripting.
Performance Considerations
Sequentum Enterprise must lock the Python GIL (Global Interpreter Lock) when executing a Python script using external Python. This means external Python scripts cannot run concurrently in a single agent session. Python scripts in different performance sessions can always run concurrently. Embedded Python does not use a GIL and therefore can also always run concurrently.
Sequentum Enterprise will often run HTML and JSON parsers concurrently, so an agent using parsers should avoid long-running external Python scripts in the parts of the agent that are processed by parsers.
An agent that only uses dynamic web browsers rarely processes any parts of the agent concurrently, so external Python scripts can often be used without a performance penalty.
Python scripts using external Python are best used in agent Initialization and Export scripts because they never run concurrently.