Citrix ADC Load Balance Config for CyberArk PSM

At the time of this post, CyberArk does not have documentation on load balancing Privileged Session Management (PSM) traffic for Citrix ADC, though they provide an example config for F5, it doesn’t translate to Citrix. Hopefully, this would help something that is trying to do this for ADC.

## Add PSM Backend Servers ##
add server PSM-01 x.x.x.x -comment cyberark
add server PSM-02 x.x.x.x -comment cyberark

## Create  RDS Health Monitor & PSM Health Monitor ##
add lb monitor CyberArk-PSMHealth-monitor HTTP-ECV -send "GET /psm/api/health" -recv PASS -LRTM DISABLED -interval 30 -resptimeout 10 -destPort 443 -secure YES -sslProfile ns_default_ssl_profile_backend
add lb monitor CyberArk-RDSHealth-monitor USER -scriptName nsrdp.pl -dispatcherIP 127.0.0.1 -dispatcherPort 3013 -LRTM DISABLED -interval 30 -resptimeout 10

## Create TCP Service Group, bind backend servers and health monitors with service group monitor threshold/weight of 2 ##
add serviceGroup CyberArkPSM-TCP3389-SG TCP -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 9000 -svrTimeout 9000 -CKA NO -TCPB NO -CMP NO -monThreshold 2 -downStateFlush DISABLED -comment "SM: CyberArk PSM Service Group"
bind serviceGroup CyberArkPSM-TCP3389-SG PSM-01 3389
bind serviceGroup CyberArkPSM-TCP3389-SG PSM-01 3389
bind serviceGroup CyberArkPSM-TCP3389-SG -monitorName CyberArk-PSMHealth-monitor
bind serviceGroup CyberArkPSM-TCP3389-SG -monitorName CyberArk-RDSHealth-monitor

## Create LB TCP VIP ##
add lb vserver CyberarkPSM_TCP3389_VS TCP x.x.x.x 3389 -persistenceType NONE -cltTimeout 9000
bind lb vserver CyberarkPSM_TCP3389_VS CyberArkPSM-TCP3389-SG

Couple of things to call out, the PSM monitor presumes the PSM Health Check backend server setup is configured with ResponseMode of Classic. Notice the PSM monitor has the destPort override of 443, without it service group port would be used causing the probe to fail. Also the monThreshold for the service group set to 2, both health checks need to pass for ADC to mark the backend server as up.

References >
Deploy PSM Health Check | CyberArk Docs
Example for configuring a load balancer | CyberArk Docs
Set up PSM high availability | CyberArk Docs

Troubleshooting > If the service group gets marked as down, start with the monitor details on the group, Last response should provide some guidence.

Sign Scripts/EXE’s with Powershell

Set-AuthenticodeSignature -FilePath .\App-Management.ps1 -TimestampServer http://timestamp.digicert.com -Certificate (Get-ChildItem cert:\CurrentUser\my -CodeSigningCert)

Note: codesign cert needs to exist under the user’s personal store. TimestampServer is recommended as the script signing would still be honored even the cert that used for signing expires in the future, timestamp server from DigiCert is used as an example, any popular public timestamp server will do.

Migrating Bookmarks/Favorites from IE11 to Microsoft Edge Chromium

The New Microsoft Edge does not support folder redirection for Favorites/Bookmarks, the AutoImportAtFirstRun policy could do a one-time import of the current IE bookmarks silently on the first launch but the challenge is to keep them in sync across various devices when users roam.

Automatically import another browser’s data and settings at first run
SOFTWARE\Policies\Microsoft\Edge\AutoImportAtFirstRun REG_DWORD 0x00000001
Group Policy (ADMX) info
  • GP unique name: AutoImportAtFirstRun
  • GP name: Automatically import another browser’s data and settings at first run
  • GP Option Mapping: FromInternetExplorer (1) = Automatically imports all supported datatypes and settings from Internet Explorer
  • GP path (Mandatory): Administrative Templates/Microsoft Edge/
  • GP ADMX file name: MSEdge.admx

https://docs.microsoft.com/en-us/deployedge/microsoft-edge-policies#autoimportatfirstrun

For the users that roam to different PCs or use VDI, Sync to Azure, or a personal Microsoft account can be enabled, this is the easiest option. However, if you like to keep the sync data local then on-premises only sync is configurable. This is similar to favorites folder redirection, with a couple of additional steps to get the same experience.

Step 1. Enable BrowserSignin

SOFTWARE\Policies\Microsoft\Edge\BrowserSignin REG_DWORD 0x00000001
Group Policy (ADMX) info
  • GP unique name: BrowserSignin
  • GP name: Browser sign-in settings
  • GP Option Mapping: Enable (1) = Enable browser sign-in
  • GP path (Mandatory): Administrative Templates/Microsoft Edge/
  • GP ADMX file name: MSEdge.admx

https://docs.microsoft.com/en-us/deployedge/microsoft-edge-policies#browsersignin

Step 2. Associate Edge profile with Active Directory (AD) account

SOFTWARE\Policies\Microsoft\Edge\ConfigureOnPremisesAccountAutoSignIn REG_DWORD 0x00000001
Group Policy (ADMX) info
  • GP unique name: ConfigureOnPremisesAccountAutoSignIn
  • GP name: Configure automatic sign in with an Active Directory domain account when there is no Azure AD domain account
  • GP Option Mapping: SignInAndMakeDomainAccountNonRemovable (1) = Sign in and make domain account non-removable
  • GP path (Mandatory): Administrative Templates/Microsoft Edge/
  • GP ADMX file name: MSEdge.admx

https://docs.microsoft.com/en-us/deployedge/microsoft-edge-policies#configureonpremisesaccountautosignin

Step 3. Enable Roaming Profile Support

SOFTWARE\Policies\Microsoft\Edge\RoamingProfileSupportEnabled REG_DWORD 0x00000001
Group Policy (ADMX) info
  • GP unique name: RoamingProfileSupportEnabled
  • GP name: Enable using roaming copies for Microsoft Edge profile data
  • GP path (Mandatory): Administrative Templates/Microsoft Edge/
  • GP ADMX file name: MSEdge.admx

https://docs.microsoft.com/en-us/deployedge/microsoft-edge-policies#roamingprofilesupportenabled

Step 4. Copy profile.pb to network share on Logout

by default profile.pb in %APPDATA%/Microsoft/Edge. Using GP logout action, copy the file to the user-specific network location.

Step 5. Copy profile.pb from network share on Login

Using GP login action, copy down the profile.pb file to %APPDATA%/Microsoft/Edge.

Add Disclaimer to Citrix ADC RfWebUI theme easily.

RfWebUI in ADC is a mobile-friendly theme, adapts fluidly to different screen layouts, and supports Citrix Receiver Extension APIs making customizing a lot easier. Moreover this is a supported approach from Citrix based on their nfactor extensibility framework.

The two files we are interested in adding a disclaimer to the login page are scripts.js and style.css. The JS file would hold the actual text and the CSS file would hold the styling code. Sample below. <RfWebUI-Custom> would be your custom theme name.

/var/netscaler/logon/themes/<RfWebUI-Custom>/scripts.js

$('.customAuthFooter').html("Disclaimer: All information, software, services, and comments provided on the site are for informational and self-help purposes only and are not intended to be a substitute for professional advice.");

customAuthFooter is an out of the box HTML ID for RfWebUI theme, others ID your might be interested are customAuthBottom and customAuthHeader HTML IDs.

/var/netscaler/logon/themes/<RfWebUI-Custom>/style.css

.customAuthFooter {
bottom: 0;
font-size: 12px;
color: white;
text-align: center;
background-color: black;
}

End Result

Invalidate staticobjects cache to see the changes right away

Note: Event if integrated cache is not enabled, gateway still uses it. Make sure to invalidate.

References >
https://www.citrix.com/content/dam/citrix/en_us/citrix-developer/documents/receiver-apis.html
https://docs.citrix.com/en-us/citrix-adc/current-release/aaa-tm/authentication-methods/multi-factor-nfactor-authentication/nfactor-extensibility.html

Update Root Certs from Windows Update when “Turn off Automatic Root Certificates Update” is Enable

Powshell snippet

$repoPath = \\<fileserver>\<share>\WindowsUpdate-RootCertsRepo
Certutil -syncWithWU -f $repoPath 
$sstStore = ( Get-ChildItem -Path $repoPath\*.crt )
$sstStore | Import-Certificate -CertStoreLocation Cert:\LocalMachine\Root

Note: avoid importing SST file using generateSSTFromWU cerutil switch as it appears to corrupt microsoft root certs. 
ref > https://social.technet.microsoft.com/Forums/en-US/13dc04f9-0f53-4071-8440-7d90d6ec9c6e/microsoft-root-certificate-authority-reported-as-revoked?forum=win10itprosecurity

Citrix ADC – Upgrade in HA Environment

Steps from bottom to top

InstanceCommand
new secondayset HA node -haSync ENABLED 
new primaryset HA node -haSync ENABLED 
new secondayRebooting Citrix ADC
new secondayVerifying task completion
new secondayInstalling Citrix ADC using build-12.1-60.19_nc_64.tgz
new secondaytar xvfz build-12.1-60.19_nc_64.tgz
new secondaycd /var/nsinstall
new secondayshell
new primaryforcefailover -f
secondayRebooting Citrix ADC
secondayVerifying task completion
secondayInstalling Citrix ADC using build-12.1-60.19_nc_64.tgz
secondaytar xvfz build-12.1-60.19_nc_64.tgz
secondaycd /var/nsinstall
secondayshell
primaryset HA node -haSync DISABLED 
secondayset HA node -haSync DISABLED 
primarysave config
primaryUploading file build-12.1-60.19_nc_64.tgz to Citrix ADC
secondayUploading file build-12.1-60.19_nc_64.tgz to Citrix ADC
secondayVerifying device backup

PowerShell code to remove DriveLetters for “System Reserved”Partition and CD-ROM

Depending on how the OS is installed, with Citrix PVS capture you may end up with “System Reserved” partition consuming a drive letter, Code below could be used during a machine startup script to provide a clean user experience. Also, included CD-ROM which I used for PVS BootISO and it is not really needed once the OS is loaded. Code validated on Server 2012 R2.

$SystemReservedDriveLetter=(Get-Volume | where {$_.FileSystemLabel -eq "System Reserved"}).DriveLetter
if ($SystemReservedDriveLetter -match  "[a-z]")
{
	mountvol "$SystemReservedDriveLetter`:" /d
}

$CDDriveLetter=(Get-Volume | where {$_.DriveType -eq "CD-ROM"}).DriveLetter
if ($CDDriveLetter -match  "[a-z]")
{
	mountvol "$CDDriveLetter`:" /d
}

Netscaler Rewrite complete HTTP Request Body before sending to the backend server

You may run into cases where the sending application has one set of message standards and the receiving service has its own standards. One such case that I recently ran into with an app trying to initiate a call request via vendor-neutral open standards to Cisco Unified communication system. Obviously, Cisco expects the API request to be in its own format and some process needs to do this translation/transformation to complete the request successfully.

Cisco has good documentation on these Call APIs, also published a sample java application which can be found at https://developer.cisco.com/site/webdialer/

Netscaler Rewrite Action can be levered to transform the HTTP REQ Body to Cisco desired format. Below is a rewrite action that I used to bridge the gap, Basically, replaced the message body to Cisco specifications from the extracted information of the original message.

Expression

Note: Had to break the expression strings in chucks of ~200 char’s to please Netscaler character limitation. Also, while I could have used XPATH to extract the desired information, I went with regex as it turned out to be faster during testing. 

"<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:urn=\"urn:WD70\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instanc"+"e\"><soapenv:Header /><soapenv:Body><urn:makeCallSoap soapenv:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\"><in0 xsi:type=\"urn:Credential\"><userID xsi:type=\"xsd:string\">username"+"</userID><password xsi:type=\"xsd:string\">******</password></in0><in1 xmlns:soapenc=\"http://schemas.xmlsoap.org/soap/encoding/\" xsi:type=\"soapenc:string\">" + HTTP.REQ.BODY(HTTP.REQ.CONTENT_LENGTH).AFTER_REGEX(re/<PhoneNumber>/).Before_REGEX(re/<.PhoneNumber>/) + "</in1><in2 xsi:type=\"urn:UserProfile"+"\"><user xsi:type=\"xsd:string\">?</user><deviceName xsi:type=\"xsd:string\">CSF" + HTTP.REQ.BODY(HTTP.REQ.CONTENT_LENGTH).AFTER_REGEX(re/<PhoneAgentID>/).Before_REGEX(re/<.PhoneAgentID>/).TO_UPPER + "</deviceName><lineNumber xsi:type=\"xsd:string\">?</lineNumber></in2></urn:makeCallSoap></soapenv:Body></soapenv:"+"Envelope>"

References >

https://www.citrix.com/blogs/2014/03/03/creating-policies-greater-than-255-characters/

https://developer.cisco.com/site/webdialer/

Extract ICA client public IP from ADM/MAS HDX INSIGHT data

If you tasked to find the public IP for all your ICA clients you will be surprised to know the Citrix monitoring/ODATA in virtual apps and desktops do not have this data.

If you have Citrix ADM/MAS in place and your ADC/Netscaler is on Premium/Platinum license, you are in luck and this could be your source for this kind of data but it doesn’t have a data export function from the web UI. This is where NITRO comes to the rescue, In fact, the Web interface for ADM/MAS gets the data through the NITRO API resources behind the scenes.

Here are some URLs that could help get the data you may seek, initially to explore the data you may start off by just navigating to these URL paths directly using any web browser. you will be prompted for credentials which would your ADM/MAS login. If it piques your interest, you may move to POSTMAN or PowerShell to automate & filter JSON responses.

Active ICA Sessions 

https://ADM_IP-ADDRESS/nitro/v1/appflow/ica_session?asc=no&pagesize=250&duration=last_1_day

Historical ICA Sessions 

https://ADM_IP-ADDRESS/nitro/v1/appflow/ica_session_inactive?asc=no&pagesize=250&duration=last_1_year

Both URLS above provide a JSON response with intresting fields eg: client_version, client_type, client_ip_address, terminated_reason etc..

For the full list of fields refer to https://github.com/citrix/mas-nitro-api-reference/blob/master/docs/configuration/analytics/hdx-insight/ica_session.md