Posted by
Share this article
In his role as Operations Manager for iQ3, Ajay Jadhav has to deal with a wide variety of technical challenges to iQ3’s Cloud Backup clients. One particular challenge is very large Microsoft Windows file servers, often 10’s of Terabytes of data, backed up in full nightly over tiny WAN links. By default the technology deployed, EMC Avamar, only allows a single backup stream per client, which can result in backups exceeding agreed windows.
A solution is Parallelism, configuring the client to run multiple instances of the backup agent, thus appearing as multiple clients in EMC Avamar, and allowing multiple parallel backup jobs to run.
WARNING: This is an advanced topic, and requires careful ongoing management as the clients and the associated directories they target should not be mixed up. It also assumes you have a solid understanding of EMC Avamar technology, the concepts of Avamar cache files, and some basic appreciation of networking.
We configured a test environment in iQ3 with four additional backup streams, targeting C, D, E, and F drives on a large Microsoft Windows client to demonstrate EMC Avamar parallelism. All names used in the commands for directories, services, etc. are completely customisable but should be kept consistent to ensure clarity.
Requirements:
- Additional Disk space for storing new Avamar \var directory & its contents
- System memory must be at least 8GB to load additional Avamar cache files during backup
- Additional firewall ports as the additional backup agents cannot listen on the same port; we recommend using 28003, 28004 etc.
Step 1
Create a new EMC Avamar VAR folder to contain configuration and cache files for each new backup agent you want to create. Keep the names aligned to the business data or directory path you intend to protect with that backup agent such as in the following example:
“C:\Program Files\avs\varC"The following image shows the end result:
Step 2
Inside the newly created “varC” folder, create a file avagent.cmd and add the below flags:
--hostname=<C-client name>
--mcsaddr=<avamar grid>
--dpndomain=<domain>
--vardir=<new var location>
--listenport=28003The hostname parameter represents the name of the client within EMC Avamar, and does NOT require DNS resolution.
Step 3
As with all EMC Avamar clients, it is necessary to “Activate” it with the new hostname to the EMC Avamar servers…but you have no system tray icon to use?
In an elevated Microsoft Windows Command prompt or Powershell, run the following command:
"C:\Program files\avs\bin\avagent.exe" --register --vardir="C:\Program Files\avs\varC"Output will be quite verbose, but you should see the highlighted text when complete:
C:\Windows\system32>"C:\Program files\avs\bin\avagent.exe" --register --vardir="C:\Program Files\avs\varC"
2018/03/29-04:19:53.39400 [avagent] Config: VARDIR=C:\Program Files\avs\varC, HOMEDIR=C:\Users\ajayj
2018/03/29-04:19:53.39499 [avagent] Looking for flag file "C:\Program Files\avs\varC\avamar.cmd"
2018/03/29-04:19:53.39599 [avagent] Looking for flag file "C:\Program Files\avs\varC\avagent.cmd"
2018/03/29-04:19:53.43899 [avagent] uflags::parsefile Printing flags from C:\Program Files\avs\varC\avagent.cmd:
.cmd flag [1]: --hostname=syd02-vs-util.iq3.com.au-c
.cmd flag [2]: --mcsaddr=syd01-g1-util.iq3baas.com
.cmd flag [3]: --dpndomain=/clients/iQ3/CORP
.cmd flag [4]: --vardir=C:\Program Files\avs\varC
.cmd flag [5]: --listenport=28003
2018/03/29-04:19:53.44000 [avagent] Looking for flag file "C:\Program Files\avs\varC\.avagent"
avagent Info <5702>: Command Line: C:\Program files\avs\bin\avagent.exe --register --vardir=C:\Program Files\avs\varC
avagent Info <5703>: Parsed Flags: C:\Program files\avs\bin\avagent.exe --hostname=syd02-vs-util.iq3.com.au-c --mcsaddr=syd01-g1-util.iq3baas.com --dpndomain=/clients/iQ3/CORP --vardir="C:\Program Files\avs\varC" --acport=28003 --initialize --vardir="C:\Program Files\avs\varC"
avagent Info <42249>: Checking for stale certificate lock
avagent Info <18908>: This client does not appear to have SSL certificates installed in the OpenSSL Certificate Store -- will try to copy from Windows Certificate Store
avagent Info <19117>: Migrating Public Certificate 'Avamar Trusted Client'
avagent Info <19124>: Public Certificate 'Avamar Trusted Client' not found in Windows Certificate Store
avagent Info <18910>: Successfully processed Windows Certificate Store and found no certificates to copy to OpenSSL Certificate Store
avagent Info <18918>: Registration: Processing secure registration with the MCS.
avagent Info <18921>: Registration: Requesting root CA from the MCS.
avagent Info <18997>: Received from MCS 100101:'Unable to retrieve rsa root or intermediate ca's. Unsecure clients only. Root-cert message rejected.'
avagent Info <18922>: Registration: MCS does not support enhanced security, using proprietary communication.
avagent Info <5933>: Checking for plug-in information files (.pin) in C:\Program Files\avs\varC
avagent Info <7324>: Volume Type for "C:\" is "NTFS", Supports Compression=1, Encryption=1, ACLS=1, DataStreams=1, Reparse=1, Sparse=1, Hardlinks=1
avagent Info <5933>: Checking for plug-in information files (.pin) in C:\Program files\avs\bin
avagent Info <5934>: Reading pin file C:\Program files\avs\bin\update.pin
avagent Info <7264>: Finished READING xml file - starting parse
avagent Info <6627>: Checking version of plug-in 'update', executable name 'avupdate' (mode:ctl)
avagent Info <5936>: Plug-in version 7.4.101-58 found for update plug-in
avagent Info <12489>: mess 0 '<plugin-description pid="update" interface="ctl" appname="avupdate" pid-ordinal="23" desc="Update plugin" exe="avupdate" use-clientlogsdir="true" version="7.4.101-58" processortype="AMD64" osversion="Windows Server 2008 R2 Standard Server Edition Service Pack 1 64-bit" osname="Windows Server 2008 R2 Standard Server Edition Service Pack 1 64-bit" />'
avagent Info <5934>: Reading pin file C:\Program files\avs\bin\vss.pin
avagent Info <7264>: Finished READING xml file - starting parse
avagent Info <6627>: Checking version of plug-in 'VSS', executable name 'avvss'(mode:ctl)
avagent Info <5936>: Plug-in version 7.4.101-58 found for VSS plug-in
avagent Info <12489>: mess 1 '<plugin-description pid="VSS" interface="ctl" appname="avvss" pid-ordinal="15" desc="Performs VSS snapup & restore" exe="avvss" version="7.4.101-58" processortype="AMD64" osversion="Windows Server 2008 R2 Standard Server Edition Service Pack 1 64-bit" osname="Windows Server 2008 R2 Standard Server Edition Service Pack 1 64-bit" />'
avagent Info <5934>: Reading pin file C:\Program files\avs\bin\windows.pin
avagent Info <7264>: Finished READING xml file - starting parse
avagent Info <6627>: Checking version of plug-in 'Windows', executable name 'avtar' (mode:ctl)
avagent Info <5936>: Plug-in version 7.4.101-58 found for Windows plug-in
avagent Info <12489>: mess 2 '<plugin-description pid="Windows" interface="ctl"
appname="avtar" pid-ordinal="1" desc="Performs Windows filesystem snapups" exe="
avtar" use-clientlogsdir="true" version="7.4.101-58" processortype="AMD64" osversion="Windows Server 2008 R2 Standard Server Edition Service Pack 1 64-bit" osname="Windows Server 2008 R2 Standard Server Edition Service Pack 1 64-bit" />'
avagent Info <6601>: Hostnamelist: syd02-vs-util01.iq3.com.au,192.168.12.253,syd02-vs-util01.iq3.com.au,192.168.222.6,vmware-localhost,127.0.0.1
avagent Info <5472>: Registering with the MCS (syd01-g1-util.iq3baas.com)
avagent Info <6610>:
****** Current MCS name 'syd01-g1-util.iq3baas.com' ******
avagent Info <7502>: Registration of client /clients/iQ3/CORP/syd02-vs-util.iq3.com.au-c with MCS syd01-g1-util.iq3baas.com:28001 successful.
avagent Info <5405>: avagent returning with exitcode 0You should now see the new client in the EMC Avamar GUI:
Step 4
While the agent configuration exists and is registered to the EMC Avamar server, there is no process or service configured to listen for that agent; again in an elevated Microsoft Windows command prompt, create the backup agent service using the following commands:
sc create backupC binpath= "C:\Program Files\test.exe" start= auto DisplayName= "C Drive" type= own
sc config backupC binpath= """"C:\Program Files\avs\bin\avagent.exe""" /ServiceStart --vardir="""C:\Program Files\avs\varC""" --logfile="""C:\Program Files\avs\varC\avagent.log""""
sc description backupC "C Backup"The output, if successful should look like this:
C:\Windows\system32>sc create backupC binpath= "C:\Program Files\test.exe" start= auto DisplayName= "Backup Agent - Drive C only" type= own
[SC] CreateService SUCCESS
C:\Windows\system32>sc config backupC binpath= """"C:\Program Files\avs\bin\avagent.exe""" /ServiceStart --vardir="""C:\Program Files\avs\varC""" --logfile="""C:\Program Files\avs\varC\avagent.log""""
[SC] ChangeServiceConfig SUCCESS
C:\Windows\system32>sc description backupC "EMC Avamar agent for Drive C only, listening on TCP port 28003"
[SC] ChangeServiceConfig2 SUCCESSIn the Microsoft Windows Services console, you should see the new services:
Start the backup agent service in the same elevated command prompt:
net start backupCStep 5
The multiple clients are now ready to run parallel backups. Either initiate them manually or configure appropriate EMC Avamar Policies to schedule the backups.
WARNING: To repeat the warning from above, this is an advanced configuration which needs to be carefully understood and managed going forward; while it is one Microsoft Windows operating system, each EMC Avamar client has:
- it’s own TCP port
- it’s own VAR directory
- it’s own configuration
- most importantly of all, it’s own set of EMC Avamar cache files. As cache files contain metadata associated with specific files and folders backed up by that client, it is strongly recommend that you always backup the same data, directories, and files with the same EMC Avamar agent.
If you would like further information about iQ3, iQ3 Cloud Backup, EMC Avamar, or other advanced backup and restore topics, please get in touch, and Ajay and his team would be more than happy to help.
Contact us
Speak with a Tesserent
Security Specialist
Tesserent is a full-service cybersecurity and secure cloud services provider, partnering with clients from all industries and all levels of government. Let’s talk.
