Sequentum Enterprise benchmarks for volumes of short runtime agents

In Q2 2021, we released an enhancement to the ACC queue that supports configuration of the interval that the ACC checks the server for available resources.  This was to optimize load balancing for volumes of short runtime agents.  Below are the tests we ran and performance we measured. 

In these tests, we used 300 agents that fetch data from News sites. Of those 300 agents, 228 agents use HTML Parser, 4 agents use JSON Parser, and the remaining 68 agents use both Dynamic Browser and HTML Parser. All 300 agents are configured to use the Exit If Data Exist command which supports download of deltas only.  In this way, agents run very quickly (often within a few seconds) since they run a frequent check for new content but only occasionally detect new content for download. 

The following table shows the list of servers and their specs that have been used for performance tests. 

Server Type vcpus RAM/GiB memory SE Version
Bare Metal Server 24 Cores Logical Processor 48 Intel (R) Xeon(R) CPU E5-2687W v4 @3.00 GHz 256 GB RAM 2.74.3
Azure Windows VM (Standard F16s_v2) 16 vcpus 32 GiB memory 2.74.3
Azure Windows VM (Standard F32s_v2) 32 vcpus 64 GiB memory 2.74.3

 

ACC  Setup

ACC Thresholds Session Delay
      sessions:40, cpu:90%, mem:90%                        Default Blank

 

SE Server Setup

Follow the Windows Heap Setting Tuning article to change the Windows Server Heap Settings.

SE Server Heap Settings SE Version
Windows SharedSection=1024,20480,4096                      2.74.3

 

Agent Settings

Internal Database Exit If Data Exist Data Validation Export Target Session Support
SQLite Yes(Internal DB) Yes JSON and AWS S3 Export Single

 

Several groups of tests were done on 3 different types of servers using the same 300 agents and SQLite as an internal database, to check the ACC Queue performance and SE performance.

Test Results

Bare-Metal Server - 24 Cores Logical Processor 48 Intel (R) Xeon(R) CPU E5-2687W v4 @3.00 GHz, 256 GB RAM 

  Test Data Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of Runs Actual No. of Runs Max Concurrent Agents

Max CPU Usage

Avg. RunTime

(seconds)

1

100 5 10 200 200 10 32% 13.76
200 5 10 400 400 21 25% 15.13
300 5 10 600 600 28 37% 16.52

 

Bare-Metal Server - 24 Cores Logical Processor 48 Intel (R) Xeon(R) CPU E5-2687W v4 @3.00 GHz, 256 GB RAM 

  Test Data Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of Runs Actual No. of Runs Max Concurrent Agents

Max CPU Usage

Avg. RunTime

(seconds)

2

100 10 20 200 200 5 12% 11.13
200 10 20 400 400 8 16% 13.00
300 10 20 600 600 16 19% 14.20

 

Bare-Metal Server - 24 Cores Logical Processor 48 Intel (R) Xeon(R) CPU E5-2687W v4 @3.00 GHz, 256 GB RAM 

  Test Data   Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of Runs Actual No. of Runs Max Concurrent Agents Max CPU Usage

Avg. RunTime

(seconds)

3

100 15 30 200 200 4 16% 11.52
200 15 30 400 400 10 22% 13.26
300 15 30 600 600 9 19% 14.05

 

Azure VM Standard F32s_v2 (32 vcpus, 64 GiB memory)

  Test Data         Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of Runs Actual No. of Runs Max Concurrent Agents Max CPU Usage

Avg. RunTime

(seconds)

4

100 5 10 200 200 8 26% 8.17
200 5 10 400 400 15 36% 10.58
300 5 10 600 600 24 66% 12.09

 

Azure VM Standard F32s_v2 (32 vcpus, 64 GiB memory)

  Test Data Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of Runs Actual No. of Runs Max Concurrent Agents Max CPU Usage

Avg. RunTime

(seconds)

5

100 10 20 200 200 4 16% 7.76
200 10 20 400 400 8 26% 10.61
300 10 20 600 600 21 51% 15.62

 

Azure VM Standard F32s_v2 (32 vcpus, 64 GiB memory)

  Test Data Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of runs Actual No. of Runs Max Concurrent Agents Max CPU Usage

Avg. RunTime

(seconds)

6

100 15 30 200 200 3 18% 9.01
200 15 30 400 400 9 56% 16.53
300 15 30 600 600 15 73% 18.72

 

Azure VM Standard F16s_v2 (16 vcpus, 32 GiB memory)

  Test Data Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of runs Actual No. of Runs Max Concurrent Agents Max CPU Usage

Avg. RunTime

(seconds)

7

100 5 10 200 200 8 30% 8.1
200 5 10 400 400 22 100% 11.72
300 5 10 600 600 22 100% 12.03

 

Azure VM Standard F16s_v2 (16 vcpus, 32 GiB memory)

  Test Data Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of Runs Actual No. of Runs Max Concurrent Agents Max CPU Usage

Avg. RunTime

(seconds)

8

100 10 20 200 200 4 18% 8.24
200 10 20 400 400 9 36% 10.73
300 10 20 600 600 16 42% 12.79

 

Azure VM Standard F16s_v2 (16 vcpus, 32 GiB memory)

  Test Data Test Results
Group No. of Agents

Schedule Interval
(mins.)

Timespan (mins.) Expected No. of runs Actual No. of Runs Max Concurrent Agents Max CPU Usage

Avg. RunTime

(seconds)

9

100 15 30 200 200 3 31% 8.74
200 15 30 400 400 6 71% 12.13
300 15 30 600 600 16 36% 13.06

 

Server: Azure VM Standard F16s_v2 (16 vcpus, 32 GiB memory)

Run Duration: 24 Hours

Schedule Frequency: Runs every 15 minutes.

  Test Data Test Results
Group Agent Count

Schedule Interval

Timespan Expected Run Count Actual Run Count

Avg. RunTime

Min.RunTime Max. RunTime
10 300 15 mins. 24 hours 300*96=28800 28800 11.85 seconds 3 seconds 265 seconds

 

Server: Azure VM Standard F32s_v2 (32 vcpus, 64 GiB memory)

Run Duration: 24 Hours

Schedule Frequency: Runs every 15 minutes.

  Test Data Test Results
Group Agent Count

Schedule Interval

Timespan Expected Run Count Actual Run Count

Avg. RunTime

Min. RunTime Max. RunTime
11 300 15 mins. 24 hours 300*96=28800 28800 11.63 seconds 3 seconds 257 seconds

 

Server: Bare-Metal Server - 24 Cores Logical Processor 48 Intel (R) Xeon(R) CPU E5-2687W v4 @3.00 GHz, 256 GB RAM 

Run Duration: 24 Hours

Schedule Frequency: Runs every 15 minutes.

  Test Data Test Results
Group Agent Count

Schedule Interval

Timespan Expected Run Count Actual Run Count

Avg. RunTime

Min.RunTime Max. RunTime
12 300 15 mins. 24 hours 300*96=28800 28800 12.44 seconds 4 seconds 153 seconds
Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.