Adding Commands Through Config
Add Navigate Link command
[FindOrAdd("ContextMenu",Link).Selection.SelectionPaths.Get(0).Set]
Xpath = //video[@class='video-stream html5-main-video']
[FindOrAdd("ContextMenu",Link).Action.Set]
IsDiscoverAction=false
ActionType=0
TargetBrowser=Current
BrowserMode=Default
[FindOrAdd("ContextMenu",Link).Action.Events.Clear(true).Set]
IsDefaultEvents=false
[FindOrAdd("ContextMenu",Link).Action.Events.Add(contextmenu).Set]
[FindOrAdd("ContextMenu",Link).Action.Events.Add(rightclick()).Set]
[FindOrAdd("ContextMenu",Link).Action.Events.Add(simulaterightclick()).Set]
[FindOrAdd("ContextMenu",Link).Action.Events.Add(rightclick).Set]
The above configuration file does the following:
Finds a Navigate Link with the name "ContextMenu". If the command does not exist, a new Navigate Link command is added.
Adds the XPath selection.
![nav1.png](../__attachments/39452673/nav1.png?inst-v=6bb97928-a461-4ed5-8611-39fb7a1182eb)
Sets the command property DiscoverAction to False.
Sets the command property ActionType to 0(FireEvents).
![logo2.png](../__attachments/39452673/logo2.png?inst-v=6bb97928-a461-4ed5-8611-39fb7a1182eb)
Sets the command TargetBrowser to Current.
Sets the command Browser to Default.
![nav2.png](../__attachments/39452673/nav2.png?inst-v=6bb97928-a461-4ed5-8611-39fb7a1182eb)
Sets the command Action Event property IsDefaultEvents to false.
Clears the default Action Events of the command.
Adds 4 Events to the command Action Events:
contextmenu
rightclick()
simulaterightclick()
rightclick.
![nav3.png](../__attachments/39452673/nav3.png?inst-v=6bb97928-a461-4ed5-8611-39fb7a1182eb)
2. Add Navigate URL command
[FindOrAdd("NavigateURLCommand",Url).Set]
ExportMethod=Separate
[FindOrIgnore("NavigateURLCommand").DataProvider.Set]
ProviderType=Simple
[FindOrIgnore("NavigateURLCommand").DataProvider.SimpleProvider.Set]
CsvData=https://labno.org/1
The above configuration file does the following:
Finds a command "NavigateURLCommand". If the command does not exist, a new Navigate URL command is added.
Sets the command property ExportMethod to Separate.
![separate.png](../__attachments/39452673/separate.png?inst-v=6bb97928-a461-4ed5-8611-39fb7a1182eb)
Sets the command property DataProvider ➤ ProviderType to Simple
Sets the command DataProvider ➤ SimpleProvider ➤ CsvData to a specific URL.
![save4.png](../__attachments/39452673/save4.png?inst-v=6bb97928-a461-4ed5-8611-39fb7a1182eb)
3. Add Manual Navigation command
[FindOrAdd("ManualNavigationCommand",ManualBrowsing).Condition.Set]
ConditionType=Script
[FindOrAdd("ManualNavigationCommand",ManualBrowsing).Condition.Script.Set]
IsEnabled = true
ScriptLanguage=Csharp
CsharpScript=[Content.ConditionScript]
[Content.ConditionScript]
using System;
using Sequentum.ContentGrabber.Api;
using System.Threading;
public class Script
{
//See help for a definition of ConditionScriptArguments.
public static bool ConditionScript(ConditionScriptArguments args)
{
//Place your script code here.
Thread.Sleep(6000);
return true;
}
}
The above configuration file does the following:
Finds a command "ManualNavigationCommand". If the command does not exist, a new Manual Navigation command is added.
Sets the command property ConditionType to use Condition Script.
Add the Condition Script to the command.
4. Add Wait for Content command
[FindOrAdd("WaitForContentCommand",WaitForContent).Selection.SelectionPaths.Get(0).Set]
Xpath=//section[@class='content']
5. Add Set Form Field command.
[FindOrAdd("FormField",FormField).Selection.SelectionPaths.Get(0).Set]
Xpath = //input[@id=”name”]
6. Add Web Content command
[FindOrAdd("Title",HtmlCapture).Set]
HtmlAttribute=Styled HTML
[FindOrAdd("Title",HtmlCapture,5).Selection.SelectionPaths.Get(0).Set]
Xpath =//h3[@class='section-title h6']
7. Adding Shared Script on a Web Content command
[FindCaptureOrIgnore("webcontent",HtmlCapture).ContentTransformationScript.Set]
IsEnabled=true
ScriptLanguage=Csharp
UseSharedLibrary=true
TemplateReference=Shared
TemplatePath=ContentTransformation\Csharp\webcontent.cgt
8. Add Page Attribute command
[FindOrAdd("Location_PageAttribute","PageAttribute",2).Set]
9. Add Download Image command
[FindOrAdd("DownloadImage",ImageCapture) ).Selection.SelectionPaths.Get(0).Set]
Xpath =//img
10. Add Download Document command
[FindOrAdd("DownloadDocument","DocumentCapture").Selection.SelectionPaths.Get(0).Set]
Xpath =//html
11. Add Download Audio command
[FindOrAdd("DownloadAudio","AudioCapture").Selection.SelectionPaths.Get(0).Set]
Xpath=//a[contains(@href,'.mp3')]
[FindOrAdd("DownloadAudio","AudioCapture").Set]
IsDirectUrl=true
DirectURL=https://download.sequentum.com/FileContent/CGTrainingDownload/audio/mp3/file_example_MP3_700KB.mp3
IsConvertAudioToText=true
Adding ConvertAudioToTextScript on Download Audio command.
[FindOrAdd("DownloadAudio","AudioCapture").ConvertAudioToTextScript.Set]
IsEnabled=True
ScriptLanguage=Csharp
CsharpScript=[Content.ConvertAudioToText]
[Content.ConvertAudioToText]
using System;
using System.IO;
using Sequentum.ContentGrabber.Api;
public class Script
{
//See help for a definition of ConvertAudioToTextArguments.
public static string ConvertAudioToText(ConvertAudioToTextArguments args)
{
return args.ScriptUtilities.SpeechToText.TranscribeAudioWithIbmWatson("https://api.{location}.{offering}.watson.cloud.ibm.com/speech-to-text/api/v1/recognize", "YOUR_KEY", args.Audio, args.AudioFormat, false);
}
}
12. Add Download Video command
[FindOrAdd("DownloadVideo","VideoCapture").Selection.SelectionPaths.Get(0).Set]
13. Add Download Screenshot command
[FindOrAdd("Screenshot","ScreenshotCapture").Selection.SelectionPaths.Get(0).Set]
Xpath = //html[head/meta[1]]
14. Add Calculated Value command
[FindOrAdd("Location_calculatedValue", "CalculatedValue").Set]
15. Add Data Value command
[FindOrAdd("Location_DataValue","DataValue").Set]
16. Add Compute Hash command
[FindOrAdd("Compute Hash","Hash").Set]
17. Add Web Element List command
[FindOrAdd("HeadingList",HtmlList).Selection.SelectionPaths.Get(0).Set]
Xpath = //h2[@class='entry-title h4']
18. Add Data List command
[FindOrAdd("Data_List","DataList").DataProvider.Set]
ProviderType=NumberRange
[Agent.FindOrIgnore("Data_List").DataProvider.NumberProvider.Set]
CsvData=https://labno.org/1
19. Add Group commands
[FindOrAdd("Group Command","Group").Set]
20. Add Group commands in Page Area
[FindOrAdd("Group_Command_in_Page_Area","HtmlArea").Set]
21. Add Branch command
[FindOrAdd("Branch","BranchIf").Set]
22. Add ExitOrRetry command
[FindOrAdd("ExitOrRetryCommand","ExitOrRetryCommand").Selection.SelectionPaths.Get(0).Set]
Xpath = //html[head/meta[1]]
23. Add ExitIfDataExist command
[FindOrAdd("ExitIfDataExist","ExitIfDataExist",0).Set]
24. Add Execute Script Command
[FindOrAdd("ExecuteScript", "CommandScript",0).Script.Set]
25. Add Transform Page command
[FindOrAdd("Transform Page","Transformation").Selection.SelectionPaths.Get(0).Set]
Xpath = //html[head/meta[1]]
26. Add Remove Duplicate command
[FindOrAdd("Remove Duplicate","RemoveDuplicate").Set]
27. Add Refresh Document command
[FindOrAdd("Refresh Document","DocumentRefresh").Set]
28. Add Close Browser command
[FindOrAdd("Close_Broswer","CloseBrowser").Set]
29. Add Download Page command as PDF Page
[FindOrAdd("DownloadPage_PDF","PdfCapture").Set]
30. Add Download Page command as an HTML Page
[FindOrAdd("DownloadPage_HTML","PdfCapture").Set]
DownloadType = Html
Adding commands at a specified position
Add Web Element List command at the first position.
[FindOrAdd("HeadingList",HtmlList,1).Selection.SelectionPaths.Get(0).Set]
Xpath = //h2[@class='entry-title h4']
2. Add Web Content command under a Web Element List command at the first position.
[FindOrAdd("HeadingList",HtmlList).FindOrAdd("Heading",HtmlCapture,1).Set]
3. Add Calculated Value command at the second position under Agent command itself
[FindOrAdd("Location_calculatedValue", "CalculatedValue", 2).Set]
4. Add Data Value command at the third position under Agent command itself
[FindOrAdd("Location_DataValue","DataValue",3).Set]
5. Add Web Content command under Agent command itself
[FindOrAdd("Title",HtmlCapture,5).Selection.SelectionPaths.Get(0).Set]
Xpath =//h3[@class='section-title h6']
6. Add Navigate Link command under Web Element List i.e. HeadingList
[FindOrAdd("HeadingList",HtmlList).FindOrAdd("Heading_NavigateLink",Link,5).Selection.SelectionPaths.Get(0).Set]
Xpath = //a
7. Add Page Attribute command under Heading_NavigateLink.
[FindOrAdd("Heading_NavigateLink",Link).FindOrAdd("Location_PageAttribute","PageAttribute",2).Set]
8. Add Navigate Url command under Heading_NavigateLink.
[FindOrAdd("Heading_NavigateLink",Link).FindOrAdd("NavigateURL",Url, 3).Set]
ExportMethod=Separate
[Agent.FindOrIgnore("NavigateURL").DataProvider.Set]
ProviderType=Simple
[Agent.FindOrIgnore("NavigateURL").DataProvider.SimpleProvider.Set]
CsvData=https://labno.org/1
9. Add Calculated Value command , change the command properties such as Change Tracking and Export Enabled and adding the Transformation Script .
[FindOrAdd("DownloadPage", "CalculatedValue", 0).Set]
IsExport=False
ChangeTracking=Exclude
[FindOrAdd("DownloadPage", "CalculatedValue",0).ContentTransformationScript.Set]
IsEnabled = true
ScriptLanguage=Csharp
CsharpScript=[Content.ContentTransformationScript]
[Content.ContentTransformationScript]
using System;
using Sequentum.ContentGrabber.Api;
using System.IO;
using System.Text;
public class Script
{
//See help for a definition of ContentTransformationArguments.
public static string TransformContent(ContentTransformationArguments args)
{
//Place your transformation code here.
//This example just returns the input data
var rawHTML = args.HtmlDocument.RawHtml;
var downloadDirectoryPath = args.Agent.DataDownloadDirectoryName;
var fileName = string.Format("{0}_{1}.html", args.Agent.Name, DateTime.Now.ToString("yyyyMMddHHmm"));
var filePath = Path.Combine(downloadDirectoryPath, fileName);
try
{
if (File.Exists(filePath))
{
using(StreamWriter sw = File.AppendText(filePath))
{
sw.WriteLine(Environment.NewLine+ rawHTML);
sw.Close();
}
}
else
{
using (FileStream fs = File.Create(filePath))
{
byte[] author = new UTF8Encoding(true).GetBytes(rawHTML);
fs.Write(author, 0, author.Length);
fs.Close();
}
}
}
catch (Exception Ex)
{
return (Ex.ToString());
}
return filePath;
}
}