Anka Build
Overview
Anka Build solution consists of following software modules.
Client software module - Anka package anka.pkg and anka agent which communicates with the controller. Controller Anka agent is installed as part of the installation of anka.pkg. Anka package is installed on the mac machines.
Server Side software modules - Controller and Registry.
CI plugins - The plugins are installed on your respective CI server (Jenkins, Teamcity etc).
Anka Client Package Install and Configuration
Installation of Anka.PKG
Install Anka package on the mac machines on which you want to execute build and test jobs. You can install Anka package on multiple mac machines. These mac machines will be nodes, when joined to the Management controller service.
Installing the Anka Hypervisor binary is remarkably simple, just download it from Anka Build Download page.
Run the anka.pkg installer. The Anka package includes the Core hypervisor, anka guest Add-ons and Controller Anka agent.
Note In order to create VMs with nested feature enabled, select nested virtualization checkbox in Customizations while installing Anka through the GUI method.
If using command line, use the following command sudo installer -applyChoiceChangesXML nanka.xml -pkg Ankaxx.pkg -target /.
After running the installer package, anka is found at /usr/local/bin/anka.
Verify the installation by running anka version command.
After install, if the users want to, they can change the location of this default Anka install location link manually.
Execute sudo mv /usr/local/bin/anka /to/any/other/location/.
Verify the installation by running anka version command.
View all default configuration settings for Anka installation on the host with anka config -l command.
anka config --help
Usage: anka config [OPTIONS] [PARAM]...
Manage Anka configuration
Options:
-l, --list List value parameter(s)
-r, --reset Reset value of parameter(s)
-s, --sorted / -S, --unsorted Enable or disable parameters output in alphabetical order
--help Show this message and exit.
anka config -l
+-----------------------------+-----------------------------------------------------------------------------+
| registry_remotes_file_path | /Users/XXX/.anka/remote |
| mac_random_bytes | 2 |
| disk_eraser_tool | /Library/Application Support/Veertu/Anka/tools/erase_disk_image.sh |
| process_type | Interactive |
| default_passwd | admin |
| nvram_path | /Library/Application Support/Veertu/Anka/uefi/anka_vars.fd |
| vm_lib_dir | /Users/XXX/Library/Application Support/Veertu/Anka/vm_lib/ |
| anka_image_maker_executable | /Library/Application Support/Veertu/Anka/bin/anka_image |
| mac_prefix_file | /Users/XXX/.anka/mac_prefix.txt |
| options_api_url | https://8ewgf8mtn4.execute-api.us-west-2.amazonaws.com/prod/key/info |
| addons_pkg | com.veertu.anka.guestaddons.pkg |
| log_dir | /Users/XXX/Library/Logs/Anka/ |
| addons_postupdate | /Library/Application Support/Veertu/Anka/tools/addons_postupdate.sh |
| vnc_password | admin |
| nice | 0 |
| product_root | /Library/Application Support/Veertu/Anka |
| addons_disk_tool | /Library/Application Support/Veertu/Anka/tools/addons_create_update_disk.sh |
| time_sync | 1 |
| max_vms_allowed | 255 |
| portfwd_base | 10000 |
| operations_timeout | 300 |
| bios_path | /Library/Application Support/Veertu/Anka/uefi/anka_code.fd |
| table_fmt | psql |
| anka_executable | /Library/Application Support/Veertu/Anka/bin/ankahv |
| block_nocache | False |
| default_user | anka |
| vm_lock_dir | /Users/XXX/Library/Application Support/Veertu/Anka/vm_lib/.locks |
| state_lib_dir | /Users/XXX/Library/Application Support/Veertu/Anka/state_lib/ |
| mac_counter | /Users/XXX/.anka/mac_counter.txt |
| uhost_executable | /Library/Application Support/Veertu/Anka/bin/uhost |
| addons_identifier | com.veertu.anka.addons.core.pkg |
| addons_image | /Library/Application Support/Veertu/Anka/guestaddons/anka-addons-mac.iso |
| ca_bundle | /Users/XXX/.anka/ca-bundle.crt |
| permanent_mac_prefix | 86,69,69 |
| anka_viewer | /Library/Application Support/Veertu/Anka/bin/AnkaView.app |
| img_lib_dir | /Users/XXX/Library/Application Support/Veertu/Anka/img_lib/ |
| puller_threads | 4 |
| allow_nonunique_names | False |
| log_file | anka.log |
| adresses_per_byte | 250 |
| trim_disk | True |
+-----------------------------+-----------------------------------------------------------------------------+
See value for a specific configuration parameter with anka config $PARAM_NAME.
Then, activate with your license key.
sudo anka license activate <key>
Note Default storage location for Anka VMs is at vm_lib_dir| /Users/XXX/Library/Application Support/Veertu/Anka/vm_lib/, state_lib_dir| /Users/XXX/Library/Application Support/Veertu/Anka/state_lib/ & img_lib_dir| /Users/XXX/Library/Application Support/Veertu/Anka/img_lib/.
Note Log directory is at log_dir| /Users/XXX/Library/Logs/Anka/ and log file is anka.log.
Managing Anka VMs on external storage on the host
If the host machines on which you install Anka has insufficient storage for Anka VMs, you can point Anka to use the external storage for the images.
There are three configuration parameters to control location for storing Anka VMs.
anka config img_lib_dir /Volumes/ExternalDrive/image_libanka config state_lib_dir /Volumes/ExternalDrive/state_libanka config vm_lib_dir /Volumes/ExternalDrive/vm_lib
You can see the full list of parameters with anka config -l.
There are different setting domains for every user, plus root. To change the root’s settings please execute the config commands with sudo.
Create VM(Recommended Method)
Anka makes it very simple to manage your macOS CI infrastructure-as-a-code.
Use anka create command to create macOS VMs from the .app installer app.
anka create --ram-size 4G --cpu-count 2 --disk-size 80G --app /Applications/Install\ macOS\ High\ Sierra.app Hisierravm
Note For Catalina Anka VMs, --ram-size value should be 4G and --disk-size should be 80G.
By default anka create creates macOS VM with administrative user - anka and password - admin. You can change this default user by using these ENV variables with anka create command.
ANKA_DEFAULT_PASSWD=passwd ANKA_DEFAULT_USER=usrname anka create --ram-size 4G --cpu-count 2 --disk-size 60G -a /Applications/Install\ macOS\ High\ Sierra.app HiSierravm
anka create --help
Usage: anka create [OPTIONS] VMNAME
Creates a VM
Options:
-m, --ram-size TEXT ram size in G [default: 4G]
-c, --cpu-count INTEGER the number of cpu cores [default: 2]
-d, --disk-size TEXT sets the disk size when creating a new disk, G/M suffix needed [default: 80G]
-a, --app PATH Path to Install macOS Application (downloadable from AppStore)
-p, --pkg PATH Additional package to be installed
-s, --postinstall PATH Postinstall scripts (to run with root credentials at first boot)
-P, --profile PATH Install configuration profile(s) alongside macOS installation
--interactive Perform default interactive installation instead of the automated one
--help Show this message and exit.
Note - anka create supports macOS manual setup. Use the --interactive flag.
Note - While creating VM with anka create, make sure to specify enough --disk-size.
anka create --ram-size 4G --cpu-count 2 --disk-size 80G --app /Applications/Install\ macOS\ High\ Sierra.app build73sierra
Installing macOS 10.13...
Preparing target disk...
Copying addons...
Converting to ANKA format...
Waiting for installation to complete in the guest (about thirty minutes approx.)...
vm created successfully with uuid: 8f0e1111-a14b-11e7-aaa4-003ee1cbb8b4
The output of anka create command is a VM created and it’s in suspended state. anka start from suspended state bypasses the full boot and starts the Vm in 1-2 seconds.
Note VMs are created with SIP/Kext Consent disabled by default. It’s strongly advised to keep these settings for optimal Anka performance.
If you need to re-enable SIP/Kext Consent, then use this command anka modify VMNAME set custom-variable sys.csr-active-config 0.
Note VMs created are in suspended mode to enable fast boot/Instant Start.
Using an ISO file to create Anka VM - For Yosemite & ElCapitan VMs Only
Use create_macos_install_image.sh included in the Anka package to first create an .iso from your Yosemite and ElCapitan .app installer.
If you already have .iso file, you don’t need to execute this step.
/Library/Application\ Support/Veertu/Anka/tools/create_macos_install_image.sh
Usage: create_macos_install_image.sh install_macos.app [OPTIONS]...
Options:
--g,--guest-addons Embed Anka guest addons in the installer
--o,--output output.iso Specify output image file, if not specified the image will be created in working directory
--p,--pkg path/to/pkg Specify additional packages to include into the installer
For example:
/Library/Application\ Support/Veertu/Anka/tools/create_macos_install_image.sh /Applications/Install\ macOS\ Sierra.app
To create a VM from .iso, you will use anka create command as you typically would. It will create an empty VM.
Note - > While creating VM with anka create, make sure to specify enough --disk-size parameter. Currently, it’s not possible to change the disk size for an existing VM.
anka create --ram-size 2G --cpu-count 2 --disk-size 60G sierravm
vm created successfully with uuid: dfaa97c5-2154-11e8-881d-acbc32ad1d59
Then, start the VM with the sierra ISO attached.
anka start -v -o sierra.iso sierravm
+-----------------------+--------------------------------------+
| uuid | dfaa97c5-2154-11e8-881d-acbc32ad1d59 |
| name | sierravm |
| cpu_cores | 2 |
| ram | 2G |
| hard_drive | 60 GB (11.2 MB on disk) |
| addons_version | not found |
| status | running |
| vnc_connection_string | vnc://:admin@10.0.1.12:5900 |
| view_vm_display | anka view sierravm |
+-----------------------+--------------------------------------+
Complete the macOS setup inside the VM. Then, stop the VM.
Start the VM again with guest addons ISO installed.
anka start -v -o /Library/Application\ Support/Veertu/Anka/guestaddons/anka-addons-mac.iso sierravm
Complete the guest addons installation inside the VM. Shutdown the VM with anka stop VMNAME.
Validate by running the following command anka run VMNAME ls -l from the host. It should display ls -l contents of the VM. The VM is correctly created.
Anka Guest Add-ons also create a default user - anka, passwd - admin for the VM.
Start VM
The VM can now be successfully started. The VM is preconfigured with a default administrative username anka and password admin. You will see the VM boot up and have to complete the macOS keypad setup steps.
anka start VMname will start the VM in headless mode.
anka start -v VMname will start the VM with the VM window.
anka start 133b387
+-----------------------+--------------------------------------+
| uuid | 49b35a9c-1659-11e8-a71d-003ee1cde439 |
+-----------------------+--------------------------------------+
| name | 133b387 (20rel) |
+-----------------------+--------------------------------------+
| description | nineteen |
+-----------------------+--------------------------------------+
| created | Jul 01 07:24 |
+-----------------------+--------------------------------------+
| cpu_cores | 4 |
+-----------------------+--------------------------------------+
| ram | 8G |
+-----------------------+--------------------------------------+
| display | 1 |
+-----------------------+--------------------------------------+
| hard_drive | 40Gi (43.8Gi on disk) |
+-----------------------+--------------------------------------+
| addons_version | 2.0.0.107 (update recommended) |
+-----------------------+--------------------------------------+
| status | running |
+-----------------------+--------------------------------------+
| vnc_connection_string | vnc://:admin@xxx.xxx.xx.xxx:5901 |
+-----------------------+--------------------------------------+
Validate by running the following command anka run VMNAME ls -l from the host. It should display ls -l contents of the host current directory. The VM is correctly created.
You can manually work within the VM with anka view sierravm. This will open the VM window.
Do anka show vmname to view IP and other runtime details of the VM.
anka show 133b387
+-----------------------+--------------------------------------+
| uuid | 49b35a9c-1659-11e8-a71d-003ee1cde439 |
+-----------------------+--------------------------------------+
| name | 133b387 (20rel) |
+-----------------------+--------------------------------------+
| description | nineteen |
+-----------------------+--------------------------------------+
| created | Jul 01 07:24 |
+-----------------------+--------------------------------------+
| cpu_cores | 4 |
+-----------------------+--------------------------------------+
| ram | 8G |
+-----------------------+--------------------------------------+
| display | 1 |
+-----------------------+--------------------------------------+
| hard_drive | 40Gi (44.1Gi on disk) |
+-----------------------+--------------------------------------+
| addons_version | 2.0.0.107 (update recommended) |
+-----------------------+--------------------------------------+
| status | running |
+-----------------------+--------------------------------------+
| ip | 192.168.64.38 |
+-----------------------+--------------------------------------+
| vnc_connection_string | vnc://:admin@xxx.xxx.xx.xxx:5901 |
+-----------------------+--------------------------------------+
port_forwarding
+--------------+------------+---------+-------------+-------------+-----------+
| guest_port | protocol | name | host_port | time_sync | host_ip |
+==============+============+=========+=============+=============+===========+
| 22 | tcp | jenkins | 10000 | 0 | 0.0.0.0 |
+--------------+------------+---------+-------------+-------------+-----------+
Managing VM Disk space
You can specify intial disk space while creating Anka Vm with anka create command. However, in some scenarios, you need to increase the disk space on an existing VM.
Change the disk space on an existing VM with anka modify vmname set hard-drive 0 <disk size> command and then execute anka run -n VM diskutil apfs resizeContainer disk0s2 <disk size>.
Upgrading macOS VM inside Anka VM
Note Don’t use the System Preference installer to update macOS inside Anka VM. Use the softwareupdate -Ri command line tool to upgrade macOS inside Anka VM. Stop the Vm and then restart it.
Working inside the VM
There are multiple methods to install various software packages and work inside the VM.
Manual method
You can manually work within the VM with anka view sierravm. This will open the VM view window.
anka view supports working in full screen and also retina mode. Retina mode is supported for Anka Vms running Mojave or later.
Automated methods
SSH to the VM and execute commands
SSH into the VM from the host where its running with the following command.
ssh anka@ip, where ip is the Vm IP shown in anka show vmname command.
To SSH into the VM from another host, first enable ssh port forwarding. Use anka modify command.
anka modify vmname add port-forwarding --host-port 0 --guest-port 22 ssh
rule added successfully
When the port forwarding rule is successfully added, you will see the following in the anka show vmname output.
port_forwarding
+--------------+------------+---------+-------------+-------------+-----------+
| guest_port | protocol | name | host_port | time_sync | host_ip |
+==============+============+=========+=============+=============+===========+
| 22 | tcp | ssh | 10000 | 1 | 0.0.0.0 |
+--------------+------------+---------+-------------+-------------+-----------+
Access it with ssh anka@hotsip -p host_port
Use anka run and execute commands
Anka command line contains anka run command (similar to docker run) to execute commands inside the VM directly from the host.
Example
anka run -n -N vmname ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
anka run -n vmname sudo gem install xcode-install
#saving user/pass of app store
echo FASTLANE_USER=user >> appstore_login
echo FASTLANE_PASSWORD=password >> appstore_passwd
anka run -f appstore_login -nE vmname xcversion install 10.1
Go to Execute operation inside Vm from the host with RUN for more details on how to use anka run command.
VM Networking
Anka VM by default use Apple’s VMNET interface for networking.
Network connectivity to the VM from out side is achieved by enabling Port forwarding for the VM.
If you want to enable ssh based access to the VM from your CI tools, then enable port forwarding.
Use anka modify command.
anka modify vmname add port-forwarding --host-port 0 --guest-port 22 ssh
rule added successfully
When the port forwarding rule is successfully added, you will see the following in the anka show vmname output.
port_forwarding
+--------------+------------+---------+-------------+-------------+-----------+
| guest_port | protocol | name | host_port | time_sync | host_ip |
+==============+============+=========+=============+=============+===========+
| 22 | tcp | ssh | 10000 | 1 | 0.0.0.0 |
+--------------+------------+---------+-------------+-------------+-----------+
In some scenarios due to network specific settings within an enterprise, You may need to cange the default starting range for port forwrding from 10000 to something else. This can be done by executing the following command on the host machine where the VM will run.
anka config portfwd_base 40000
If this node/host is connected to the controller execute this command in system domain, since controller operates in system domain.
sudo anka config portfwd_base 40000
Accessing the host from within Anka VM
The host from Anka VM has fixed IP address of 192.168.64.1 (or 192.168.128.1 in host-only mode).
*Note Make sure to execute this on all the nodes/hosts.
Anka Client CLI Commands
Anka package provides an complete set of CLI interface to manage macOS infrastructure as a cloud for CI purposes.
Execute anka help to see a complete list CLI commands available.
anka --help
Usage: anka [OPTIONS] COMMAND [ARGS]...
Options:
--machine-readable JSON output format
--log-level [debug|info|error]
--debug
-l, --login TEXT Specify the vm policy user (if configured)
--help Show this message and exit.
Commands:
clone Clones a VM
config Manage Anka configuration
create Creates a VM
delete Deletes a VM (or list of vms)
describe Shows VM configuration
license licensing commands
list List VM library contents
modify Modifies a VM settings
mount Mounts local folder into VM
reboot Restarts a VM(s)
registry VMs registry
run Run commands inside VM environment
show Shows VM runtime properties
start Starts or resumes paused VM
stop Shuts down a vm
suspend Suspends a VM(s)
unmount Unmount shared folder (filesystem...
usb Do actions on USB devices
version prints out version
view Open VM display viewer
machine readable output from Anka CLI Commands - Use --machine-readable flag. Example anka --machine-readable clone VMname Vmname2.
Debugging Anka CLI Commands - Use --debug flag. Example anka --debug clone VMname Vmname2.
Create VM - Use anka create command. Go to Create VM section.
Clone VM - Use anka clone command. -c flag consolidates/shrinks all the snapshots and creates an independent copy.
anka clone --help
Usage: anka clone [OPTIONS] VM_ID NEW_VM_NAME
Clones a VM
Options:
-c, --copy Create independent copy instead of clone
--help Show this message and exit.
Manage Mac host Anka Disk Space - If you have created multiple versions or tags of a VM and pushed and pulled them on the host, then Anka caches delta associated with the tags to optimize future pulls from the registry. This can sometimes lead to excessive disk utilization on the Mac host. Use anka clone -c to consolidate/shrink all the snapshots and create an independent copy of the VM.
Manage VM configuration - Use anka config command to manage VM configuration.
anka config --help
Usage: anka config [OPTIONS] [PARAM]...
Manage Anka configuration
Options:
-l, --list List value parameter(s)
-r, --reset Reset value of parameter(s)
--help Show this message and exit.
Delete VM - Use anka delete command to delete one, multiple or all VMs in the local VM directory.
anka delete [OPTIONS] [VMID]...
Deletes a VM (or list of vms)
Options:
--yes flag. don't ask - just delete
-a, --all Delete all vms in library
--help Show this message and exit.
Show VM configuration - Use anka describe command to view Vm configuration.
anka describe --help
Usage: anka describe [OPTIONS] VM_ID
Shows VM configuration
Options:
--help Show this message and exit.
Manage anka license - Use anka license command to activate, remove and execute other licesing related operations for the host.
anka license --help
Usage: anka license [OPTIONS] COMMAND [ARGS]...
licensing commands
Options:
--help Show this message and exit.
Commands:
accept-eula accept EULA (root privileges)
activate activate license key (root privileges)
remove removes the current license (root privileges)
show show license information
validate validates the current license
Note To see core consumption information for your license, use anka license show <licensekey>
list of all VMs - Use anka list command to view all the VMs available in the local VM directory with their status and other properties.
anka list --help
Usage: anka list [OPTIONS] [VMID]...
List VM library contents
Options:
-r, --running show only running vms
-s, --stopped show only stopped vms
-f, --field TEXT configure fields for each VM to be shown. Supported fields: name, version, uuid, creation_date,
status, stop_date, description, policy, cpu_cores, cpu_frequency, cpu_htt, ram, display.
--help Show this message and exit.
Modify VM properties - Use anka modify command to change VM properties like port-forwarding, vCPU, ram, etc. You can set, add and delete properties.
SET Operations
anka modify vmname/ID set --help
Usage: anka modify set [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
cpu set number of cpu cores and frequency
custom-variable configure variables
description set textual description of the VM
display configure displays
hard-drive modify hard drive settings
name set new name for the VM
nested enable nested virtualization
network-card modify network card settings
policy Enable VM access management (available in Anka Secure license)
ram set RAM size and parameters
Change the disk space on an existing VM with anka modify vmname set hard-drive 0 <disk size> command and then execute anka run -n VM diskutil apfs resizeContainer disk0s2 <disk size>.
Changing hardware properties for a VM
Use anka modify set custom-variable command. You can set the following custom varibales.
- boot-args - control the corresponding NVRAM variable
- hw.uuid - specifies the Hardware UUID
- hw.serial - specifies the Serial Number (system)
- hw.manufacturer - SMBIOS parameter (Reserved)
- hw.product - SMBIOS parameter (Reserved)
- hw.family - SMBIOS parameter (Reserved)
- hw.board - SMBIOS parameter (Reserved)
anka modify VM set custom-variable hw.UUID "GUID"
anka modify VM set custom-variable hw.serial 'MySerial'
ADD Operations
sudo anka modify vmname/ID add --help
Usage: anka modify add [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
hard-drive
network-card
optical-drive
port-forwarding
usb-device (available in enterprise and enterprise plus tiers)
Anka also supports bridge networking configuration for the VMs. The default is shared (VMNet based).
anka modify vmname/ID add network-card --help
Usage: anka modify add network-card [OPTIONS]
Options:
-t, --type [shared|host|bridge|disconnected]
[default: shared]
-b, --bridge TEXT
-m, --mac TEXT
--help Show this message and exit.
DELETE Operations
sudo anka modify vmname/ID delete --help
Usage: anka modify delete [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
custom-variable
hard-drive
network-card
optical-drive
policy
port-forwarding
usb-device (available in enterprise and enterprise plus tiers)
Mount local file system into the VM - Use the anka mount command to mount current local folder from the host inside the VM. Returns 231 on timeout.
anka mount --help
Usage: anka mount [OPTIONS] VM_NAME [DIRS]...
Mounts local folder into VM
Options:
--help Show this message and exit.
Unmount mounted folder - Use the anka unmount to Unmount the locally folder mounting. Returns 231 on timeout.
anka unmount --help
Usage: anka unmount [OPTIONS] VMID [DIR]...
Unmount shared folder (filesystem pass-through)
Options:
-a, --all unmount all mounted folders
Reboot VM - Use the anka reboot to reboot one, multiple or all Vms in the VM local directory.
anka reboot --help
Usage: anka reboot [OPTIONS] [VMID]...
Restarts a VM(s)
Options:
-f, --force flag. restarts the vm process
-a, --all reboot all running vms
--help Show this message and exit.
Work with registry - Use the anka registry command for access, push, pull Vms from the registry. See the Manage VM Templates in Registry Manage VM Templates in Registry documentation.
Start VM - Use the anka start coammnd to start VM. use the -u flag to update the guest addons inside the VM. This maybe be required for upgrading Vms for major Anka releases.
anka start --help
Usage: anka start [OPTIONS] VM_ID
Starts or resumes paused VM
Options:
-u, --update-addons, --update Run guest addons update procedure, previous version of the addons should be already
installed
-f, --force Start VM with minimum checks
-v, --view Open display window
-o, --optical-drive TEXT Path to ISO file or device
-d, --usb TEXT USB device ID/Location (should be claimed)
--help Show this message and exit.
Stop VM - Use the anka stop command to stop VM.
anka stop --help
Usage: anka stop [OPTIONS] [VMID]...
Shuts down a vm
Options:
-f, --force force the vm process to shut down
-a, --all Shutdown all running vms
--help Show this message and exit.
Suspend VM - Use anka suspend command to put Vm in Instant Boot state. It’s recommended to suspend VMs before pushing them to the Registry for use in CI.
anka suspend --help
Usage: anka suspend [OPTIONS] [VMID]...
Suspends a VM(s)
Options:
-a, --all suspend all running vms
--help Show this message and exit.
Open Vm Window/viewer - Use anka view command to open running VM window. anka viewer supports Retina support for Mojave VMs.
anka view --help
Usage: anka view [OPTIONS] VM_ID
Open VM display viewer
Options:
-d, --display INTEGER Specify the displays to view
-s, --screenshot Make png screenshot
--help Show this message and exit.
View Anka version - Use anka version command to view current anka version.
anka version
Anka Build Enterprise version 2.0 (build 108)
Show running VM properties - Use anka show command to view VM properties of running VM.
anka show --help
Usage: anka show [OPTIONS] VM_ID [PROPERTY]...
Shows VM runtime properties
Options:
--help Show this message and exit.
Attach and Detach USB devices to VM - Use anka USB command to attach and work with USB devices inside the VM. The USB devices should be connected to the host. See working with real devices.
Anka RUN Command
anka run command provides a container like interface to work with the Anka Vms directly from the host where they are running.
Execute operation inside Vm from the host with RUN - Use anka run coammnd, similar to docker run to execute operation inside the Vm from the host where it’s running. Returns 125 on timeout.
anka run --help
Usage: anka run [OPTIONS] VM_NAME COMMAND [ARGS]...
Run commands inside VM environment
Options:
-w, --workdir PATH Working directory inside the VM
-v, --volumes-from, --volume PATH
Mount host directory (current directory by default) into VM . '--volumes-from' is
deprecated form
-n, --no-volumes-from, --no-volume
Use this flag to prevent implicit mounting of current folder (see --volume
option). '--no-volumes-from' is deprecated form
-E, -e, --env Inherit environment variables. '-e' is deprecated form
-f, --env-file PATH Provide environment variables from file
-N, --wait-network Wait till guest network interface up
-T, --wait-time Wait for guest time sync
--help Show this message and exit.
Examples of anka run usage
anka run sierrav40c1 xcodebuild -sdk iphonesimulator -scheme Kickstarter-iOS build will mount the default directory from the host into the sierrav40c1 Vm and execute build.
anka run -w /Applications VMNAME ls -l will pipe the results of ls -l from the VM’s /Applications directory.
anka run -v . VMNAME ls will mount the host current directory inside the VM, execute ls, pipe the results and unmount.
anka run -v . VMNAME xcodebuild ... will mount the current directory from the developer machine(host) to the VM and execute an xcodebuild command and pipe the results back.
anka run sudo ... executes commands inside the VM with sudo privileges. For instance:
anka run VMNAME cp -R simpledir /Users/anka will copy the current host directory to the VM at /Users/anka location
anka run -n sierrav40c1 xcodebuild -sdk iphonesimulator -scheme Kickstarter-iOS build will not mount the curent host directory and execute build in the VM current directory
$ anka run VMNAME sudo whoami
root
It’s also possible to explicitly specify mount directory inside VM, using this syntax -v /host/forlder:/mnt/path, e.g:
anka run -v $PWD:/tmp/mountpoint_1 VMNAME pwd
Currently only single volume could be specified in the run command. If you need more than one folder shared between host and the VM, please use anka mount/unmount commands.
anka mount VMNAME ~/Library/MobileDevice/Provisioning\ Profiles/ /Users/anka/Library/MobileDevice/Provisioning\ Profiles/
anka run VMNAME xcodebuild -exportOptionsPlist exportInfo.plist archive
Please note, VM should be running for mount command. If you need more configuration changes to the VM, before anka run execution, you could start the VM with corresponding parameters, or write the parameters to VM configuration with anka modify command.
Working with environment variables using anka run
anka run command doesn’t use .profile, .bash_profile by itself. To configure the environment variables, you can pass it with anka run -f environment.txt, where environment.txt is a text file in the form VARIABLE=VALUE, one variable per line.
To inherit entire host’s environment, use anka run -E command, but in this case existing guests variables will not be overridden by host’s variables.
In order to use environment variables from VMs .bash_profile, execute the following command via intermediate bash.
anka run ios-temp bash -c 'source ~/.bash_profile; ./iPhone/make_build -b dev -c 1 -d ./build -e Mobile -z'
Few things to note
anka run doesn’t support TTY mode, but you could easily use POSIX streams as with regular bash tool.
anka run -n VNMANE whoami > /dev/null
cat file.txt | anka run -n VMNAME md5
anka run, when run in default mode with no flags, mounts the current folder from the host into the VM and executes commands on this mount point as current directory.
Joining/Disjoining Anka Clients to the Controller
Joining Anka Client
Anka Clients (Mac running Anka.PKG) are called Anka Nodes. Multiple nodes can be attached to the Controller to configure Anka Build Cloud.
Pre-Requisites - Anka Controller should be configured and running before an Anka Node can be joined.
Note Make sure Autologin and neversleep option are set on Anka mac nodes.
This step adds the mac hosts running Anka Client package to the Anka Build cloud Controller.
Execute this command on the mac hosts running Anka package with Build license - sudo ankacluster join <http://controllerIP:port>.
You can also set other parameters like name(the name that shows up in controller dahsboard for this host), max-vm-count (number of concurrent VMs to run) etc. See the full list of flags available with this command.
When joining the nodes using ankacluster join, use sudo. Anka is multi user environment like unix. When you join with sudo, controller will start VMs with sudo. You can view them with sudo anka list and operate with these VMs using sudo with other anka commands.
Note - Starting from version 1.2.0, ankacluster command will now ask to be run with sudo. So ankacluster join http://<host> will return available flags and a error message. It’s possible to override this behaviour with the -f or the --force-no-sudo flag.
Usage:
sudo ankacluster join [controller_address] [flags]
Flags:
-c, --cacert PATH PATH to CA bundle file (PEM/X509) (optional)
-M, --capacity-mode string Capacity mode (resource|number). 'resource' will take vm jobs according to resources, number will take vm jobs according to max-vm-count (default "number")
-C, --cert PATH PATH client certificate file (PEM/X509) (optional)
-K, --cert-key PATH PATH client certificate key file (PEM/X509) (optional)
-f, --force-no-sudo Force the agent to start without sudo
-g, --global DEPRECATED! Install agent into system domain (optional)
-G, --groups string Specify group name (or names sepearated by ',') to add this node to (optional)
-h, --help help for join
-H, --host string Specify host name or IP for this machine (optional)
-k, --keystore PATH PATH to certificate and keystore (PEM, PKCS12) (optional)
-p, --keystore-pass PASSWORD PASSWORD for certificate and keystore (optional)
-m, --max-vm-count NUMBER Maximum NUMBER of VMs this node is allowed to run (optional) (default 2)
-n, --name NAME Node NAME alias (optional)
--no-central-logging Dont send logs to central logging (optional)
-R, --ram-override int Override the max RAM in GB that this node can handle. defaults to (total ram - 2GB)
-r, --root-cert PATH PATH to CA bundle file (PEM/X509) (optional)
--skip-tls-verification Skip TLS verification (optional)
-t, --tls Use tls for communicating with the queue (optional)
-V, --vcpu-override int Override the max vcpu that this node can handle. defaults to (cpu count * 2)
Note - New in version 1.2.0 - Control the concurrent VM count execution on a node using a static number of dynamic capacity available. For dynamic capacity, use the -M, --capacity-mode string in conjuction with -m, --max-vm-count NUMBER and -R, --ram-override int flags. Default is based on static Number and is 2.
Note - In version 1.2.0, -g flag is deprecated but backward compatibility is supported.
Note - In version 1.3, controller will auto shutdown any running unmanaged VMs under sudo anka list. For any manual maintainence of VMs under sudo, disjoin the node first.
Note - In version 1.3, the anka agent running on the nodes sends logs to the central controller. This is useful for debugging purposes. This can be disabled with --no-central-logging during ankacluster join command.
Disjoining Anka Client
This step will remove the node from the Anka Build cloud. Execute this when you want to do maintainence on the node etc. You don’t need to disjoin nodes to upgrade Anka on the node.
sudo ankacluster disjoin
Note - In version 1.2.0, -g flag is deprecated for the disjoin command.
Anka Client Upgrade
Upgrading Anka Package
- Before upgrading, you will also need to force stop any suspended VMs with the
anka stop -f vmnamecommand. - Download the latest version of Anka package (Anka.pkg).
- Install the new package.
Note For major Anka releases, it maybe required to upgrade guest addons in existing Anka VMs. Check the release notes to identify if this step is required or not.
- Upgrade guest addons on the VM. Execute the
anka start -u vmnameto upgrade guest addons in existing VMs to the current release. If you are not able to upgrade the guest add-ons tool using theanka start -u vmnamecommand, then you have a very old version of guest addon tools on your VM. You will first need to manually update them. Contact Veertu support. - Push your upgraded VMs to the Registry.
anka stop -f vmname
anka start -u vmname
Preparing update configuration
Installing updates
Suspending
update succeeded
Anka Client Uninstall
Uninstall
From the command line, execute the following command. Make sure all your VMs are stopped.
sudo /Library/Application\ Support/Veertu/Anka/tools/uninstall.sh
/Library/Application\ Support/Veertu/Anka/tools/uninstall.sh --help
Usage: uninstall.sh [OPTIONS...]
Options:
-a|--along-with-machines Also remove all Anka virtual machines (for all users) created and license information
-f|--force Force mode (do not ask user confirmation)
-h|--help Show this message
Note Uninstall doesn’t remove the license. To remove license execute sudo anka license remove before uninstalling. Make a note of the fulfillment id that is displayed in the output of sudo anka license remove.
Moving license from one host to another
To move the license from an already activated mac to another mac, execute the following steps.
sudo anka licenser remove. Then, make a note of the fullfillment id shown in the output of this command. Contact Veertu support with the activation key and the fulfillment id.
Server Side - Controller/Registry Install and Configuration
Introduction
Registry and Controller services are required to configure an Anka Build private macOS cloud. Registry and controller are packaged in two flavors.
- Linux (CentOS 7) Docker container package - anka-controller-registry-XXX.tar.gz
- Mac Application package - AnkaControllerRegistry-XXX.pkg
Download the package that you want to install from Anka Build Download page
Default setup installs controller and registry together. You can configure them to run on two separate linux instances and/or mac.
Ports Information
CI system calls the Anka Controller over http or https, default port is 80.
Anka Controller does not communicate with Anka Build nodes. Nodes communicate with controller through a Queue service(installed as part of controller).
The default port for Anka registry is 8089. All the default port and other configuration settings can be modified.
Controller/Registry Mac Application package
Install
Install the AnkaControllerRegistry-XXX.pkg on the mac machine on which you want to install controller and Registry.
Default settings
- Anka Controller listens on port 80
- Anka Registry is listening on port 8089
- Registry data is stored at
/Library/Application Support/Veertu/Anka/registry
Start,Stop Controller
Use sudo anka-controller command to start/stop/check status of the controller.
sudo anka-controller --help
usage: /usr/local/bin/anka-controller [start|stop|restart|status|logs]
Changing location of VM storage in the Registry
- Stop the controller service
sudo anka-controller stop - Change the default settings, including the location of registry storage by editing the
/usr/local/bin/anka-controllerd. - Start the controller service
sudo anka-controller start
Go to the localhost:port in the browser on the mac where you installed the controller/registry package and you will see the controller dashboard.
Uninstall
From the command line, execute the following command.
sudo /Library/Application Support/Veertu/Anka/tools/controller/uninstall.sh
Controller/Registry Linux Application package
Install
Note You can run Controller/Registry docker containers on a linux instance. We don’t recommend that you run these containers on Mac hardware. If you want to run controller/registry on mac, then use the mac application package.
Move the anka-controller-registry-XXX.tar.gz tar file to a directory where you want the containers to run.
- Uncompress the tar package.
- Edit
config.ymland enter values for the following : external_registry_address, data_mount_dir. Note default port settings - Controller on port 80 and registry on port 8089. - Run
./ankaCloudConfig docker fromFile. This will update your docker-compose with configuration changes. - Execute
docker-compose up -d --build - Execute
docker ps -ato check if the anka controller and anka registry containers are up and running. - Go to controllerIP in your web browser and check to make sure that you can access the controller portal dashboard.
Note If you want to change default port settings, don’t edit config.yml. Edit the advanced.yml.
advanced.ymlconfiguration settings to edit for custom ports - external_registry_address, listen_on, data_mount_dir.- Run
./ankaCloudConfig docker fromFile -i advanced.yml - Execute
docker-compose up -d --build
Upgrade
First, Download the latest anka-controller-registry-XXX.tar.gz from Anka Build Download page.
Then, execute the following steps.
Go to the controller/registry directory and execute
docker-compose stop.Backup your existing
config.ymlandadvanced.yml.Uncompress the new controller/registry tar package.
Replace
config.ymland/oradvanced.ymlwith your backedup copies of these files.Execute
./ankaCloudConfig docker fromFileand/or./ankaCloudConfig docker fromFile -i advanced.ymlExecute
docker-compose up -d --build
Uninstall
- Go to the controller/registry directory and execute
docker-compose stop. - Delete the controller/registry directory.
Working with the Registry
Anka registry provides an easy way to store, version, and distribute macOS VMs that are used for CI and development. Once you’ve completed creation and setup of your VMs, use anka registry command to work with the registry.
Store your build and test VM templates in the registry.
Then, you can pull them, modify them and push them again with a different tag to the Registry. This way you can maintain versions. Pull specific VM template and tag to a different machine running Anka package.
Registry operations are available through Anka Client command line and through Registry REST APIs.
Working with registry from Anka Client
View all of the available registry commands by running anka registry --help.
anka registry --help
Usage: anka registry [OPTIONS] COMMAND [ARGS]...
VMs registry
Options:
-r, --remote TEXT Use repository name instead of 'default'
-a, --registry-path TEXT Use repository URL instead of 'default'
-c, -i, --cert, --pem PATH Client certificate if required
-k, --key PATH Private key, if the client certificate doesn't contain one
--cacert, --root-cert PATH Alternate ca_bundle
--insecure Skip TLS verification
--help Show this message and exit.
Commands:
add Add repository
check-download-size Returns size of the data to be downloaded
delete Forget repository
describe Shows VM description
list List VMs in repository
list-repos List registry repositories
pull Pull certain VM
push Push VM version (tag) to repository
set Set default registry
Add registry to a Node/machine running Anka - anka registry add <givesomename> <registryURLwithport>
You can add/define multiple registries to a node. The last one added is treated as current. To change current use the set command.
Set Current registry - anka registry set <previouslydefiniedname>
Push VM to the Registry - anka registry push -d <description> -t <tag> vmname
Pull VM from the Registry - anka registry pull -t <tag> vmname
Pull VM from the Registry with Shrink - anka registry pull -s -t <tag> vmname. For example, let’s say you have V1, V2 tags in the Registry for a VM. You pull V2. Then, you pull V1 with -s flag. It will optimize the local disk space usage by deleting all V2 related tag/version files.
Push independent VM as new tag of existing VM in Registry - anka registry push NEW_VM -v EXISTING_VM_IN_REGISTRY -t NEW_TAG. For example, let’s say you have a VM1, with latest tag t2. Now, you want to push a completely independent VM2 as the next tag to VM1. You will use anka registry push VM2 -v VM1 -t NEW_TAG.
List VMs in the Registry - anka registry list
Working with Registry REST APIs
To list all VM templates stored in the registry
url= "/api/v1/registry/vm"
method="GET"
body=none
Returns: Operation result, List of registry VM UUIDs and names
To show information on a specific VM template from the registry
url= "/api/v1/registry/vm?id={vm_id}"
method="GET"
body=none
Returns: Operation result, VM UUID, name and tag list
To show vm config file for a specific version or tag (or latest if none specified) of a VM template
url= "/api/v1/registry/vm?info"
method="GET"
Arguments:
id - string, the template id to get (required)
tag - string, get info for a specific tag (optional)
Version - int, get info for a specific version
If neither version or tag_name is supplied the latest version will be deleted
Return:Json, the vm config
Body Keys:
ram
hard_drives
usb
uuid
network_cards
firmware
state_file
name
nvram
version
cpu
display
Example Response
{
"message" : "",
"status" : "OK",
"body" : {
"nvram" : true,
"state_file" : "668e52df4d454f0fbc14cc69a15c5006.ank",
"display" : {
"frame_buffer" : {
"height" : 768,
"pci_slot" : 29,
"vnc_ip" : "0.0.0.0",
"width" : 1024,
"password" : "admin"
}
},
"version" : 1,
"usb" : {
"pci_slot" : 7,
"host" : 0,
"tablet" : 1
},
"network_cards" : [
{
"mode" : "shared",
"pci_slot" : 28,
"mac_address" : "56:45:45:3c:db:02",
"port_forwarding_rules" : [
{
"host_port" : 0,
"protocol" : "tcp",
"rule_name" : "ssh1",
"guest_port" : 22,
"host_ip" : "0"
}
]
}
],
"hard_drives" : [
{
"pci_slot" : 4,
"controller" : "ablk",
"file" : "239bb374545141ceb481d495ec01683e.ank"
}
],
"cpu" : {
"cores" : 2
},
"name" : "appium-base",
"ram" : "2G",
"uuid" : "2fa0f10e-e91e-4665-8d42-00a39b9707de",
"firmware" : {
"type" : "uefi"
}
}
}
To delete a specific VM template and all associated tags from the registry
url= "/api/v1/registry/vm?id={vm_id}"
method="DELETE"
body=none
Returns: Operation result
To delete a specific VM template-tag and all later tags from the registry. For this call use the IP:port of the registry.
Url: "/registry/revert?id={vm_id}&tag_name={tobedeletedtag}"
Method: DELETE
Arguments:
vm_id - must
Version - the version number to delete, all newer versions will be deleted as well
Tag_name - specify tag name instead of numeric value
If neither version or tag_name is supplied the latest version will be deleted
Return:
{“statusâ€: “OKâ€} / {“statusâ€: “errorâ€}
To distribute a specific VM template to all build nodes. Note Group_id is only available if you are running Enterprise tier of Anka Build.
url= "/api/v1/registry/vm/distribute"
method="POST"
body="VM Uuid, tag, version, group id"
group_id (optional) distributes the image to a specific group
Example Body: {"template_id":"226d946b-2467-11e7-84b7-a860b60fd826", “tageâ€=â€v1â€}
Returns: Operation result, request id
To get distribution status - Not supported in current version. Known issue with this API.
url= "/api/v1/registry/vm/distribute?id={request_id}"
method="GET"
body=none
Returns: Operation result, map of node id -> (distribution status, template id, tag, version, time)
Working with the Controller
Anka Controller is the central management system of Anka Build and provides a simple and extensible interface for provisioning and managing on-demand macOS VMs on a cluster of mac hardware (Anka Build nodes). If you use CI tools like Jenkins, Teamcity. GitLab CI, BuidKite, integrate them with Anka controller using plugins or controller REST APIs to provision macOS VMs on-demand for CI job requests.
You can work with controller through the web portal interface, REST APIs or the CI plugins.
Controller Portal
Access it by going to your controller IP - http://<controllerIP>:port.
You can view the status of your Anka Build macOS cloud from this UI and also perform basic management operations.
Dashboard View
This view displays the total active build nodes, running VM instances, instance run capacity utilization, registry storage consumption, average cpu and ram utilization across the entire cloud.
Nodes View
Click on nodes to go to node list view.
You can view all active build nodes, instances running on them, their cpu and ram utilization. From this view, you can modify the concurrent VM capacity for each node.
Templates View
Click on templates to look at all VMs stored in the registry.
Click on an individual template to view all versions/tags for that template.
Click on distribute to all nodes or select specific Nodes to pre-load the most frequently used Vm templates for your build/test jobs on all build nodes. This will reduce the time for first time job execution on Anka Build cloud. Controller manages disk space on the Build Nodes and deletes VM templates that are not used for CI build and test jobs.
Note - Once a job executes on a build node in a specific VM, the original VM template used for this job is cached on the node. Hence, any subesequent job executions don’t download VM from the registry. The VM template is only downloaded when there is a brand new VM template or a new tag to an existing VM template. Download of a VM template with a new tag is only incremental.
Distribution to nodes is complete.
Instances View
Click on Instances to get a list of all running instances on the cloud.
Manually starting instances
Click on create instance to manually start instances using a specific VM template/tag on the cloud.
Accessing Error logs
Starting from Controller release version 1.0.12, logs will be available for download from the Controller Management portal for error scenarios during VM provisioning.
Controller REST APIs
Use the REST APIs to integrate Anka Build cloud with your CI system(If there is no plugin/integration available).
To provision and start VMs (Start VM instance will start N count of instances of a particular type across the node cluster)
Note Group_id, priority and USB_device is only available if you are running Enterprise and higher tier of Anka Build.
url= "/api/v1/vm",
method="POST"
body="VM UUID, tag, version, count, node_id, startup_script, group_id, priority, usb_device"
(node_id, count, tag, version, startup_script, group_id, priority, usb_device are optional. If tag is not provided, the latest version will be used, otherwise whatever tag is specified will be pulled on the nodes. By default count is 1. If node_id is not specified the VM could be scheduled on any node)
startup_script is base64 encoded string of the script to be executed after the instance is started
Group_id lets you choose on which group the vm will run on
Priority lets you give the vm priority over other vms, most urgent priority is 1 and the default priority is 1000
Returns: Operation Result, Array of instance UUIDs
Example Body: {"vmid":"226d946b-2467-11e7-84b7-a860b60fd826", “countâ€=10, startup_script=â€IyEvYmluL2Jhc2gNCmV4cG9ydCBBPSJhYWEiDQoNCg==â€}
To terminate VM
url= "/api/v1/vm",
method="DELETE"
body=Instance ID
Example Body: {"id":"7ed72625-ee2e-4195-606a-dd04a274b9c3"}
Returns: Operation result
To list the VM instances
url= "/api/v1/vm"
method="GET"
body=none
Returns: Operation result, List of initiated VM instance UUIDs
To show information for an instance
url= "/api/v1/vm?id={instance_id}"
method="GET"
body=none
Returns: Operation result, State, VM and connectivity info
To list all build nodes joined to the controller
url= "/api/v1/node"
method="GET"
body=none
Returns: Operation result, List of nodes IDs
To show information for a build node
url= "/api/v1/node?id={node_id}"
method="GET"
body=none
Returns: Operation result, State and Info of the node and available USB devices
To update node configuration
url= "/api/v1/node/config"
method="POST"
body: JSON
Keys:
‘node_id’: string, the node id (required)
‘capacity-mode’: string, the new capacity mode (number|resource)
‘vcpu_override’: int, override the default vcpu scheduling limit
‘ram_override’: int, override the default ram scheduling limit
Example Request Body:
{"node_id": "524c3781-0035-45fe-8d98-4d6bad7d79a7", "capacity_mode": "resource"}
To delete/remove a build node from controller database/portal.
Note This doesnot remove the node from the cluster. Use ankacluster disjoin on the node to disjoin the node from the cluster. Then, execute delete/remove node API call to remove it permanently from the controller database.
url= "/api/v1/node"
method="DELETE"
body=node_id ID
Example Body: {"node_id":"7ed72625-ee2e-4195-606a-dd04a274b9c3"}
Returns: Operation result
To create request to save image to the registry connected to the controller
url= "/api/v1/image"
method="POST"
body=
{
"id": (string) the instance id to save
"target_vm_id": (string) the target template to save the image to (optional, required if new_template_name not supplied)
"new_template_name": (string) save the image as a template with this name (optional, required if target_vm_id not supplied)
"tag": (string) the tag name to use for the push
"description": (string) the description to use for the push (optional)
"suspend": (boolean) if true, suspends the vm before push (optional, defaults to true)
"script"
"revert_before_push": (boolean) when supplying 'target_vm_id' revert the latest tag of the template or tag specified in 'revert_tag' (optional, defaults to false)
"revert_tag": (string) revert this specific tag, if tag does not exist revert does not happen (optional)
"do_suspend_sanity_test": (boolean) if suspend is true, perform a suspend sanity test before pushing to the registry (optional)
}
Returns: json
{
“status”: “OK”,
“body”: save_request_id ,
“message”: error message if not “OK”
}
To get status on save image request
Path: api/v1/image
Method: GET
Query Parameters: “id” (returns only the single request)
Returns: json
Save image request format:
{
“id”: the request’s id,
“status”: pending/done/error,
“request”: an object that represents the request,
“error”: error message if status is error
}
If id supplied:
{
“status”: “OK”,
“body”: single ‘Save image request’,
“message”: error message if not ‘OK’
}
If id not supplied:
{
“status”: “OK”,
“body”: [] list of ‘Save image requests’,
“message”: error message if not ‘OK’
}
To manage ankacluster join flag which enables/disables node sending anka agent logs to the controller
Path: /api/v1/node/config
Method: POST
Body: json
{
“node_id”: the node’s id,
"disable_central_logging": boolean (true to stop, false to start/resume)
}
Returns:
{
“status”: “OK”,
“message”: “” error message if not ‘OK’
}
Curl example:
curl -X POST "192.168.1.100:80/api/v1/node/config" -d '{"node_id": "9759b4ba-b0bf-4346-b72a-d8b5ff794873", "disable_central_logging": false}'
Note - Group and USB management APIs are only available in Enterprise and higher Anka Build Tier.
To get a list of all groups definied
List Groups
url= "/api/v1/group"
method=â€GET"
body=none
Returns: Operation result, List of groups
To create a new Group
url= "/api/v1/group"
method="POST"
body="Name, Description"
Example Body: {"name": "My awesome group", "description": "my favorite nodes"}
Returns: Operation result, The new group
To update a Group
url= "/api/v1/group?id={group_id}"
method=â€PUT"
body=â€Name, Descriptionâ€
Example Body: {"name": "My other awesome group", "description": "my second favorite nodes"}
Returns: Operation result
To delete a Group
url= "/api/v1/group?id={group_id}"
method=â€DELETE"
body=none
Returns: Operation result
To add nodes to a Group
Add Nodes To Groups
url= "/api/v1/node/group"
method="POST"
body="Groupd Ids, Node Ids"
Example Body: {"group_ids": ["a2a833a1-e44c-4247-57d1-6cd06b0fd040", "47f0a57e-9b14-4d80-4f5c-3284f2c85e0d"], "node_ids": ["c6493f47-7106-4cb9-88be-1f5cf2af7a72"]}
Returns: Operation result
To remove nodes from a group
url= "/api/v1/node/group"
method="DELETE"
body="Groupd Ids, Node Ids"
Example Body: {"group_ids": ["a2a833a1-e44c-4247-57d1-6cd06b0fd040", "47f0a57e-9b14-4d80-4f5c-3284f2c85e0d"], "node_ids": ["c6493f47-7106-4cb9-88be-1f5cf2af7a72"]}
Returns: Operation result
To get a list of all USB devices attached to the cloud
List all USB devices
url= "/api/v1/usb"
method=â€GET"
body=none
Returns: Operation result, List of all USB devices attached across all the nodes
Debugging Issues with Controller
Anka controller is a server that usually runs as a daemon and errors are usually written to a log.
For mac based installation
The easiest way to debug the controller is to run ‘/usr/local/bin/anka-controllerd’ directly. This will output the controller log to the screen and you will be able to see if there are any errors.
1. First stop the controller with sudo anka-controller stop.
- Then execute
/usr/local/bin/anka-controllerd.
You can also tail the log, the default location is ‘/Library/Logs/Veertu/AnkaController’. Check ‘/usr/local/bin/anka-controllerd’ in order to see the log dir location.
For docker based installation
Check the output of docker ps. The anka controller container status should be something like this.
“Up for x amount of time”. For example:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ae6b522fdec4 test220919_anka-controller "/bin/bash -c 'anka-…" 2 minutes ago Up 2 minutes 0.0.0.0:8090->80/tcp test220919_anka-controller_1
If the controller is crashing the status could be “restarting” or “failed”. Example for restarting state: “Restarting (1) 12 seconds ago”.
You can try to run docker-compose up without any parameters and this will output stdout for all services to the screen.
In case one of the services is crashing you could see the errors.Here is an example.
anka-controller_1 | E1128 15:17:16.952500 1 main.go:618] Could not build tls configuration:
anka-controller_1 | E1128 15:17:16.953204 1 main.go:619] open /whwhwhw/path: no such file or directory
test220919_anka-controller_1 exited with code 1
You can also take a look at the logs using docker logs --follow $CONTAINER_NAME.
Integrate Jenkins with Controller
The Anka Jenkins plugin provides a quick way to integrate Anka Build Cloud with Jenkins for iOS/macOS CI workflows. This enables Jenkins jobs to dynamically provision specific macOS VM instances(based on label used) on Anka Build Cloud for job execution. The VMs are deleted after every successfull job execution. New job request starts a brand new VM instance on the Anka Build Cloud.
Anka Jenkins plugin supports both Pipeline and Freeform Jenkins jobs.
NOTE : Starting from version 1.20 of Anka jenkins Plugin, the Slave template builder plugin features are integrated in Anka Jenkins plugin. It’s possible to uninstall Slave template builder plugin, install the Anka jenkins plugin version 1.20 and configure cache builder settings similar to Slave Template builder configuration.
Preparing the VM Template
- Install openJDK 8 in your macOS VM Template.
- Create jenkins User(required for ssh based connectivity from Jenkins to Anka VMs). Make sure remotelogin is enabled for this user. You can also use default
ankauser. - Enable port forwarding (required for ssh based connectivity from Jenkins to Anka VMs).
- Suspend the VM.
- Push it to anka registry.
SSH or JNLP based Connection from Jenkins to Anka VMs
Anka Build Jenkins plugin supports both JNLP and SSH based connections to Anka VMs.
JNLP - For JNLP, configure for JNLP in the Anka Build plugin in Jenkins.
SSH
- Create a dedicated (Jenkins) user and enable remote login for this user in the VM template. Make sure to enable auto login for the administrative user of the VM as well.
- Configure port forwarding
Configure the Anka Cloud Plugin in Jenkins
Install the Anka Build Cloud Plugin - Browse to
Manage Jenkins > Manage Plugins.Click on
Available. Search for Anka and you can install it from jenkins Plugin center or click on theAdvancedtab. Use the instructions under the Upload Plugin section to upload theanka-ci.hpiplugin file to your Jenkins master server.Go to
Manage Jenkins > Configure Systemand find the section called Cloud. Enter a name for your Anka Cloud in the Anka Build Cloud field, as well as an IP address and port number for the controller in the Build Controller URL field(http://xx.xxx.xxx.xxx:portno). Default port is 80.Click Show Templates and configure Jenkins slave templates for your jobs.
Select the VM from the Templates. This will show the list of all VMs from the registry.
Select the tag/version from Template Version Tag dropdown. Leave it to Latest if you want to always access the latest tag/version of the VM.
Leave # of Executors to 1.
Enter the Jenkins user home path in Remote FS Root.
Enter a value for Labels that you will refer to in your jobs.
Select SSH or JNLP method for connection between jenkins and Anka VMs.
Enter value for Slave name template. Vms provisioned for this slave template definition will have this value.
You can also pass Environment Variables.
Select a Node Group if you want Vms for this definition provisioned on a specific group definied in your Anka Build Cloud (Avaialble only in Enterprise and Enterprise Plus Tiers).
Enter Priority for Vm provisioning (Available only in Enterprise and Enterprise Plus tiers).
Apply and Save the settings.
Note You can save multiple slave VM template profiles with a unique label name corresponding to different VM template types/tags. Then, your jobs can refer to these labels.
Configure the Cache Builder Section(Optional)
This section integrates the Jenkins Slave template Builder Plugin functions into the Anka jenkins plugin, to eliminate the need to install two plugins.
Configure this section if you want to pre-populate the VM templates with build caches through jenkins job, save them back to the registry and use them for your mainline jobs.
Target Template : This is the template to push the cache populated VM back to. Most of the times, it will be the same VM template you used to build caches in.
Tag : Specify the tag you want to push the cache populated VM template with. If left empty, it will take the jenkinsjobname and append current date/time to it.
Description : Decription for the VM being pushed.
Suspend : Select, if you want to push the Vm in suspended or stopped state
Delete Latest Tag : Delete the latest tag from the registry before pushing this tag. This will enable optimization of registry disk space, of the cache builder job is run frequently.
Wait For Build Finish : When using pipeline, a step or stage might finish before the build has a result. The plugin uses the build result as the condition for saving the image. In case there is no result yet, the plugin will push the template. You can use ‘currentBuild.result’ in your build step to mark the job as failed. Defaults to false. Note - Do not use Wait for Build Finish option in combination with ankaGetSaveImageResult, as it can result in a deadlock.
Pipeline Usage for cache builder section Pipelines can have multiple agents running in one build (also in parallel). The plugin relies on the build result in order to tell if it need to execute the “save image request” or not (the request will only be sent if the build succeeded).
Pipeline stages can be executed and done before the build has a result, so we can use one of two solutions.
Mark the build from the pipeline As explained here: https://support.cloudbees.com/hc/en-us/articles/218554077-How-to-set-current-build-result-in-Pipeline It is possible to set ‘currentBuild.result’ from within the pipeline itself. With this option if the build will be marked as “FAILED” “ABORTED” or “UNSTABLE” The save image request will not be issued
Wait for the build to finish Check “wait for build finish” in the slave template cache builder section. When this checkbox is checked the slave will be kept alive until the build attached to it gets a result.
Post Build Step to check status of Anka Image Save action
Fail build: Will set the build as failed if the image save request failed (or timed out). If this is not selected, the post build step will do nothing.
Timeout: The timeout is set in minutes and should be long enough to wait for the image push to registry be complete. If the value is very small, it will generate a false timeout error.
Using Post Build Step to check status of Anka Image Save action in Pipeline
Use the ankaGetSaveImageResult Pipeline step.
ankaGetSaveImageResult will return true if all previous image save calls for this particular build have been successfully executed.
shouldFail: Type - Boolean value. Fails the job if one of the image save requests has failed or timed out (default: true)
timeoutMinutes: Type - Integer in minutes. Stops waiting for the result of the template update after x minutes. (default: 120)
ankaGetSaveImageResult will return true immediately if no calls have been made.
NOTE If “Wait For Build Finish” is marked for the label’s Cache Builder configuration, this step will identify no image saving requests and thus will return true immediately.
Using Jenkins Plugin Cache Builder With Pipeline
Overview
A build runs on a VM provisioned by the plugin.
When the build is finished (in case of success) the plugin will issue a “save image” request to the controller instead of terminating the VM.
The VM will then be pushed to the registry defined in the controller configuraton.
The build will get the result of the push using ankaGetSaveImageResult.
Follow the earlier steps to configure the plugin and the cache builder section.
Example Pipeline
Scripted Pipeline example
def label = "CacheVM"
node(label){
stage('Build Cache') {
try {
// scm pull…
sh 'sleep 20' // do whatever
// error("simulated error!")
} catch (Exception e) {
currentBuild.result = 'FAILURE' // fail the build on any error
}
}
}
ankaGetSaveImageResult shouldFail: true, timeoutMinutes: 120
When you run this example you should see the instance in “pushing” state as shown below.
When the build is finished the console output will be as follows.
Uncomment the ‘error’ row in the earlier Scripted pipeline example to simulate error in pipeline. Now, the build will fail and the template is not pushed.
Note ‘ankaGetSaveImageResult’ will still output “Checking save image status… Done!”, since there is no push operation to wait for.
General observations about using Cache Builder
How often should I cache? - The answer of course depends on the size of your data, and the density of your builds.
You should run your cache build once, check how much time the operation takes. The push can be a few gigabytes big (at least 2 GB when using suspend) and might take some time on slower networks.
Note that only one template tag can be pushed at a time. You can execute multiple cache builds for the same template but those requests will end up executed in a serial manner.
If your cache build is fast - you should consider running cache builds a few times a day, or based on commits.
If your cache build is slow - Consider running one cache build per day (you can schedule it for a time that jenkins is not very busy).
The cache build should have a job or pipeline of it’s own. Caching after every “regular” build might not make sense as the time that it takes to download code or artifacts is usually the same or shorter than the time it takes to push a registry to the vm.
Creating Dynamic Label
This section describes the steps to create dynamic label for an AnkaNode. Use createDynamicAnkaNode function.
The pipeline function/step
createDynamicAnkaNodestarts a vm and returns a label. The returned label can be used as an agent in the node agent part. Dynamic Anka nodes has all the configuration options that a static Anka slave template has, with slight changes. The configuration options are passed to the function as parameters. This allows you to have only a minimal anka cloud configuration and define Anka nodes on the fly.You can check out the examples on this page - https://github.com/veertuinc/jenkins-dynamic-label-example. For a helper form, search for
createDynamicAnkaNodeon the path http://your.jenkins.server/pipeline-syntax on your jenkins server.
*** Parameters for CreateDynamicAnkaNode Function ***
| Name | Type | Default Value | Decription | Required |
|---|---|---|---|---|
| masterVmId | String | the uuid of the VM template to run | Yes | |
| tag | String | VM template tag | ||
| remoteFS | String | Users/anka | Remote node workspace | |
| launchMethod | String | jnlp | Node launch method ‘ssh’ or ‘jnlp’ | |
| credentialsId | String | Id of jenkins credentials object for ssh | ||
| extraArgs | String | String to be appended to jnlp command | ||
| javaArgs | String | String to append to jnlp java args | ||
| jnlpJenkinsOverrideUrl | String | Override the jenkins server url. Jenkins url is taken from jenkins configuration | ||
| jnlpTunnel | String | Jnlp tunnel to use for node launcher | ||
| keepAliveOnError | boolean | false | Keep the instance alive on case of build error | |
| timeout | int | 1200 | Timeout for starting the instance (in seconds) | |
| environments | List of tuples | List of environment variables to add for the build. environments: [[name: ‘FOO’, value: ‘BAR’], [name: ‘OR’, value: ‘IS’]] | ||
| nameTemplate | string | Template to use for vm and node name. You can use $template_name, $template_id, $ts | ||
| priority | int | Override the default priority (lower is more urgent) | ||
| saveImage | boolean | false | Save the vm as an image instead of terminating it | |
| suspend | boolean | false | When saving image suspend the vm before the push | |
| templateId | string | When saving image, push the vm to this template | ||
| pushTag | string | When saving image, a string to use as tag. A timestamp will be appended | ||
| deleteLatest | boolean | false | When saving image, delete the latest version before pushing | |
| templateDescription | string | When saving image, use the description for the new tag | ||
| group | string | Group id to start the instance to (only available in enterprise) | ||
| numberOfExecutors | int | 1 | Number of jenkins executors to run on this node (recommended 1) | |
| description | string | Description for this template | ||
| labelString | string | Override the returned label (not recommended) |
Note - Example Code
def LABEL = createDynamicAnkaNode(masterVmId: 'e1173284-39df-458c-b161-a54123409280')
pipeline {
agent {
label "${LABEL}"
}
stages {
stage("hello") {
steps {
sh "echo hello"
}
}
}
}
In this example, the pipeline uses the “LABEL” variable just as a regular string label would be used. The function call
createDynamicAnkaNode(masterVmId: 'e1173284-39df-458c-b161-a54123409280')will start an instance with the template id ‘e1173284-39df-458c-b161-a54123409280’, using the jnlp method. The remaining node properties will have their default values as mentioned in the table above. The command will block until the instance is up and the node is connected to Jenkins. The function will then return a generated label. The generated label will assure that no other job would use that specific node.
Integrate TeamCity with Controller
The Anka TeamCity plugin provides a quick way to integrate Anka Build Cloud with TeamCity for iOS/macOS CI workflows. This enables TeamCity jobs to dynamically provision specific macOS VM instances on Anka Build Cloud for job execution. The VMs are deleted after every successfull job execution. New job request starts a brand new VM instance on the Anka Build Cloud.
Preparing the VM Template
- Install openJDK 8 in your macOS VM Template.
- Create Teamcity User(required for ssh based connectivity from Teamcity to Anka VMs). Make sure remotelogin is enabled for this user. You can also use default
ankauser. - Enable port forwarding (required for ssh based connectivity from Teamcity to Anka VMs).
- Download TeamCity agent executable in the Vm under the newly created TeamCity user.
- Go to
BuildAgent/conf/buildagent.propertiesunder Teamcity user in the VM and check all the settings. Make sure that serverurl pointing to your Teamcity server IP asserverUrl=http://xx.xxx.xx.xxxis correct. - Start the Teamcity agent service inside the VM with
sh BuildAgent/bin/mac.launchd.sh load. - Connect to this VM from Teamcity, wait for few minutes for Teamcity to connect and authorize the VM agent instance. It will show up under the connected tab in Agents in TeamCity.
- Suspend the VM with
anka suspend VMName. - Push it to anka registry.
Configure the TeamCity Plugin
Install the Anka Build Cloud Plugin - Browse to
Administration > Plugins List. Upload the anka-build-tc-XX.zip.Create Cloud profile for the project and select
Anka Build Cloudfor Cloud Type.Edit the Cloud profile to complete configuration.
For Additional terminate conditions, select
after first build.For Controller URL, enter Anka Build Cloud controller IP and port. Default port is 80.
Image Name will show all VM templates from the Registry. Select the one you want to use for build/test for this project.
Image Tag will show all tags for the selected VM template in the Image Name field. Select the one you want to use for your build/test. Leave it to latest to always select the latest one.
For Group (optional), if you want Vms for this definition provisioned on a specific group definied in your Anka Build Cloud (Avaialble only in Enterprise and Enterprise Plus Tiers).
SSH User and SSH Password are the VM Teamcity user ssh credentials.
Agent Path Select is the path to TeamCity agent location in the VM.
Select a Node Group if you want Vms for this definition provisioned on a specific group definied in your Anka Build Cloud (Avaialble only in Enterprise and Enterprise Plus Tiers).
Enter Priority for Vm provisioning (Available only in Enterprise and Enterprise Plus tiers).
Save the settings.
Integrate GitLab CI with Controller
Veertu provides and maintains a Gitlab CI Runner for Anka Build. Teams who are using GitLab CI to execute their iOS CI pipelines can dynamically provision macOS Vms on their Anka Build cloud.
Get more details at Anka Build GitLab CI Integration Download page.
Integrate Buildkite CI with Controller
Teams who are using BuikdKite to execute their iOS CI pipelines can dynamically provision macOS Vms on their Anka Build cloud with the BuildKite integration.
Get more details at Anka Build Buildkite Integration Download page.
Packer Builder for Anka
Refer to Anka Packer Builder Download page.
Using real devices attached to Anka VMs for CI
If you run tests on real devices connected to your mac machines, then you can replicate the same setup on Anka Build macOS private CI cloud with USB features available in the enterprise tier.
Attach real devices to Anka Build node host mac machines through the USB interface.
Establish trust between the device and the host. To access iOS devices from the VM the device and VM have to be trusted.
The trust procedures are the same in the VM and host, but is the device is not physically detached from the USB port, connection to VM could be treated as a security attack, and the device won’t respond to VM’s requests. In this case, it’s recommended to not “trust” host on iOS devices before it’s done for VM. If the host is already trusted before passing the iOS device to VM, it’s recommended to clear the pairing information by Settings->General->Reset->Reset Network Settings. For a more detailed explanation see this blog.
Claim the devices using
anka usb claimcommand. If you have multiple device types like iPhone7, iPhone8, Apple watch then claim each type across your Anka Build node host mac machines under the same name. For example, claim all iPhone7 devices under claim name “iPhonedevice” using theanka usb claim -nflag, Apple watch under “watchdevice”. Then, use this claim name along with VM start command to get access to a VM with the device attached.Use the controller REST API call
url= "/api/v1/usb"to get a list of all real devices/USB devices connected to the Anka Build cloud.Start VMs on-demand with a real device attached using command
anka start -dor start from the controller REST API usingurl= "/api/v1/vm"and pass the USB_device variable.
anka USB commands
Claiming, listing and releasing USB attached devices on Anka Hosts/nodes
anka usb --help
Usage: anka usb [OPTIONS] COMMAND [ARGS]...
Do actions on USB devices
Options:
--help Show this message and exit.
Commands:
claim make a device(s) available for attaching to a...
list list available usb devices
release release a device(s) back to host availability
Claiming devices with same claim name, -n flag, to create a virtual group across multiple hosts/nodes
anka usb claim --help
Usage: anka usb claim [OPTIONS] [DEVID]...
make a device(s) available for attaching to a vm
Options:
-n, --claim-name TEXT claim name could be used as additional name
--help Show this message and exit.
Starting a VM with device attached at runtime using the -d flag
anka start --help
Usage: anka start [OPTIONS] VM_ID
Starts or resumes paused VM
Options:
-u, --update-addons, --update Run guest addons update procedure, previous
version of the addons should be already
installed
-f, --force Start VM with minimum checks
-v, --view Open display window
-o, --optical-drive TEXT Path to ISO file or device
-d, --usb TEXT USB device ID/Location (should be claimed)
--help Show this message and exit.
Anka Build Tier Datasheet
| ** ** | Basic | Enterprise | Enterprise Plus |
|---|---|---|---|
| Core based licensing | Yes | Yes | Yes |
| Cloud Controller with REST APIs | Yes(Single instance of Anka controller included) | Yes(Single instance of Anka controller included) | Yes |
| Central Registry | Yes(Single instance Anka Registry included) | Yes(Single instance Anka Registry included) | Yes |
| Jenkins Anka Cloud plugin | Yes | Yes | Yes |
| TeamCity Plugin | Yes | Yes | Yes |
| Gitlab Ci Plugin | Yes | Yes | Yes |
| BuildKite Plugin Support | Yes | Yes | Yes |
| HA for Controller configuration setup | Yes (Additional controller/registry instances needed) | Yes (Additional controller/registry instances needed) | Yes |
| Priority scheduling of VMs through controller | Yes | Yes | |
| Clustering (Grouping) of Nodes | Yes | Yes | |
| USB support (to connect real devices to the VM) | Yes | Yes | |
| Basic controller authentication, also includes REST APIs (Superlogin user) | Yes | Yes | |
| Multi-user authorization support through SSO support | Yes | ||
| Controller API activity logging | Yes | ||
| VM full disk encryption for Build VMs | Yes | ||
| Control VM runtime (Networking, Access to host) and functional properties with policies | Yes |
Enterprise Features
Anka Build Enterprise License tier enables additional features on top of Build Basic Tier.
Clustering(Grouping) - Enterprise and Enterprise Plus This feature enables users to group Anka Build nodes into clusters/groups. These clusters can then be specified selectively to provision VMs. When nodes are put in groups, the level of separation is provided at the controller level for VM provisioning requests initiated from the controller.
For example, you can create group A, group B. Then, as required some CI job requested VMs can only be provisioned on Group A or Group B.
A fallback group can be defined for any group. The fallback group will pick up the VM provisioning request if the primary group is running at its capacity.
Managing group definitions - Can be done through controller Portal UI or Controller REST APIs.
Joining Anka Build nodes to the group - Can be done at the time of joining the node, through Controller REST APIs, Controller REST APIs.
Jenkins and TeamCity Anka Plugins expose grouping feature.
Priority Scheduling - Enterprise and Enterprise Plus Priority parameter is exposed through “Create VM†Controller REST API and also through the controller portal UI. This parameter enables priority provisioning of the VM, when there are multiple requests in the controller queue.
Range - 1 - 1000. 1 - Indicates the most urgent. 1000 - Default for all
USB support - Enterprise and Enterprise Plus Attach one or multiple devices (through the host USB interface) to dynamically provisioned VMs for testing. Exposed through controller REST APIs and command line interface.
Basic controller authentication also includes REST APIs (Superlogin user) (Available from Controller 1.1 release) - Enterprise and Enterprise Plus
Authentication support includes Root token authentication access to the Controller Dashboard and certificate authentication for following clients - Build nodes, plugins, API access, anka command line access to the registry.
Enterprise Plus Features
Anka Build Enterprise Plus License tier enables additional features on top of Build Basic and Enterprise Tiers.
Authentication and Authorization support for multi-users through SSO (Available from Controller 1.1 release) - Enterprise Plus Multi-user access authentication and authorized based access to Controller portal dashboard and REST API operations is provided through OpenID and LDAP SSO based integration.
VM encryption of Build VMs at REST - Enterprise Plus Encrypt the build VM template at the time of VM creation, store it in the Anka Registry in an encrypted state. When this VM is used to build on the Anka Build nodes, it will be decrypted.
VM Control through Policy - Enterprise Plus Manage the Build VMs access to local and remote resources through policies. This includes the ability to control VM access to host shared folders, USB, disk access, networking, external networking.
Securing Anka Build Cloud
Basic Tier
In the basic tier, you can use TLS to encrypt communication between Anka Build components.
| TLS based encryption | Details |
|---|---|
| Anka Controller and Registry have built-in TLS support, configured through command-line arguments (or docker-compose.yml) | What is encrypted - Communication between controller and dashboard, registry and dashboard, registry and Anka running on Build nodes/host mac machines |
| Plugins to the controller | What is encrypted - Communication between plugins and the controller REST APIs |
Enterprise Tier
Enterprise Tier supports authentication on top of TLS.
This tier supports 2 levels of authentication.
- Root token authentication
- Client Certificate authentication
This tier has no concept of authorization, so every authenticated connection is treated as if it has same permissions.
| Authentication Method | Details |
|---|---|
| Root token authentication | Should be used to configure authenticated superuser access to the Anka Portal |
| Certificate authentication | Access from Anka host machines(for registry), Nodes(both registry and controller), Plugins, Controller and registry API access |
Enterprise Plus Tier
Enterprise Tier supports SSO based authorization and authentication on top of TLS.
This tier supports 3 authentication methods and the concept of authorization.
- Root token authentication
- Client Certificate authentication
- OpenId connect authentication(SSO)
| Authentication Method | Details |
|---|---|
| Root token authentication | Superuser access to Anka controller’s dashboard |
| Certificate authentication | Access from Anka host machines(for registry), Nodes(both registry and controller), Plugins, Controller and registry API access |
| OpenId connect | Permission-based Anka Controller Portal Dashboard access |
Note It may be possible to use Openid connect for API access through service accounts depending on the Openid provider.
How To Anka Controller and Registry have built-in Certificate, Root token and Openid connect authentication support, configured through command-line arguments (or docker-compose.yml).
Authorization
Each user of Anka Controller has two properties - username and groups.
Each group has a list of permissions that the users that belong to it are allowed to perform.
Groups can be configured using the admin UI, given that the current user has permissions to configure them or the user is authenticated using the root token (in which case the user is a “superuser”).
The authorization in the controller and registry performs checks against the groups the user belongs to.
For example:
- Group “devops” has the “list_vms” permission
- Group “admins” has the “start_vm” permission
User Bob is a member of “devops” and “admins”. When user Bob logs in to the controller portal, Bob will be able to list vms and start vms.
The authorization in tlsProxy performs checks against a static list of groups and/or usernames given by command line parameters (or in docker-compose.yml).
For example:
Node mac_mini_1 is member of group “nodes”, tlsProxy is configured with groups “nodes”, “testers”, “admins”, Node mac_mini_1 is allowed to connect to tlsProxy
Another example :
Node mac_pro_1 is member of group “admins”, tlsProxy is configured with groups “nodes” and names “mac_pro_1”, “mac_mini_1”, Node mac_pro_1 is allowed to connect to tlsProxy
Securing Anka VMs
Enterprise Plus tier includes features to encrypt VMs at REST and control VM properties like networking, copy/paste, etc through VM policy.
Certificate based authentication for Controller
Getting or Creating a CA certificate
For this tutorial you will need a CA certificate with it’s private key. For more information about CAs - https://en.wikipedia.org/wiki/Certificate_authority.
For the rest of this section CA certificate and key will be referred to as ‘ca-crt.pem’ and ‘ca-key.pem’. If you don’t have one you can create one with openssl.
openssl req -new -nodes -x509 -days 365 -keyout ca-key.pem -out ca-crt.pem -subj "/CN=myOrganization"
This command will create two files: ca-crt.pem and ca-key.pem.
Configuring Server TLS
The first step will be to configure tls for the server. The server certificates are not part of the authentication process and doesn’t need to be derived from the CA. This means that you can use certificates supplied by a 3rd party like “let’s encrypt” or use the certificates your organization supplies.
If you do not have tls certificates for your server, you can create them now. The following commands will create server certificates for a machine with a certain ip. Replace $MACHINE_IP with your server’s ip.
openssl genrsa -out controller-key.pem 4096
openssl req -new -nodes -sha256 -key controller-key.pem -out controller-csr.pem -subj "/CN=*" -reqexts SAN -config <(cat /etc/ssl/openssl.cnf <(printf "[SAN]\nsubjectAltName=IP:$MACHINE_IP"))
openssl x509 -req -days 365 -in controller-csr.pem -CA ca-crt.pem -CAkey ca-key.pem -CAcreateserial -out controller-crt.pem -extfile <(echo subjectAltName = IP:$MACHINE_IP)
For Mac Controller Setup
Edit /usr/local/bin/anka-controllerd in the following manner.
Change
LISTEN_ADDRESS=":80"
To
LISTEN_ADDRESS=":443"
Search for the row that looks like this
"$CONTROLLER_BIN" --listen_addr "$LISTEN_ADDRESS" --log_dir "$LOG_DIR" --data-dir "$DATA_DIR" $ANKA_REGISTRY $STANDALONE $ALLOW_EMPTY_REGISTRY
This is the controller’s run command. Append the following parameters to this line.
--server-cert $PATH_TO_SERVER_CERT --server-key $PATH_TO_SERVER_KEY --use-https
For Docker Controller Setup
You need to mount a directory containing the certificates on the docker container. Create a directory and copy your certificates to that directory.
Edit docker-compose.yml. In the controller’s section add a volume pointing to the directory you just created mounted on “/mnt/cert”. (we will use /home/ubuntu/certs as an example).
Also change the ports forwarding to 443 (https).
anka-controller:
build:
context: .
dockerfile: anka-controller.docker
ports:
- "443:80"
# To change the port, change the above line: - "CUSTOM_PORT:80"
volumes:
# Path to ssl certificates directory
- /home/ubuntu/cert:/mnt/cert
Now let’s add the path to our certificate and key. Search for the environment variables “USE_HTTPS”, “SERVER_CERT” and “SERVER_KEY” in docker-compose.yml. Edit them so they will look like the configuration below. If they do not exist - add them. Note that if you named your files differently the values for these parameters should match them (if you named your certificate my-crt.crt the SERVER_CERT value should look like ‘/mnt/cert/my-crt.crt’).
anka-controller:
build:
context: .
dockerfile: anka-controller.docker
ports:
- "443:80"
# To change the port, change the above line: - "CUSTOM_PORT:80"
volumes:
# Path to ssl certificates directory
- /home/ubuntu/certs:/mnt/cert
depends_on:
- etcd
# - beanstalk
restart: always
environment:
USE_HTTPS: --use-https
# Server certificate pem
SERVER_CERT: --server-cert /mnt/cert/controller-crt.pem
# Server private key pem
SERVER_KEY: --server-key /mnt/cert/controller-key.pem
Test the configuration
Start or restart your controller and test the new TLS configuration.
Point your browser to the hostname or ip serving the controller dashboard using ‘https://’. You can also try using curl -v https://$HOST/api/v1/status -k.
If that doesn’t work, try to repeat the steps and validate that the file names and paths are correct. If you are still having trouble, debug the system as explained in the Debugging controller section.
Using client certificates
The controller’s authentication module uses the CA file passed to configuration to validate the client certificates. The file should contain one (or more) certificate authority that was used to create the client certificates.
When the client will send requests to the controller, it will present it’s certificates. Those certificates will then be validated against the configured CA.
The common name of the client certificates will be used as the user name and the organization fields will be used as groups (this is more relevant for authorization in enterprise plus version).
Creating client certificates
You can use the following openssl commands to create client certificates.
openssl genrsa -out node-key.pem 4096
openssl req -new -sha256 -key node-key.pem -out node-csr.pem -subj "/CN=my_node/O=nodes"
openssl x509 -req -days 365 -in node-csr.pem -CA ca-crt.pem -CAkey ca-key.pem -CAcreateserial -out node-crt.pem
This assumes that your CA certificate and key are in the same directory and called ‘ca-crt.pem’ and ‘ca-key.pem’ respectively. If it’s not, use the correct path.
Configuring the controller with the CA file and enabling authentication
For Mac Controller Setup
Edit /usr/local/bin/anka-controllerd.
The last line should be the controller’s run command and should look similar to this (it might have your server certificate from before).
"$CONTROLLER_BIN" --listen_addr "$LISTEN_ADDRESS" --log_dir "$LOG_DIR" --data-dir "$DATA_DIR" $ANKA_REGISTRY $STANDALONE $ALLOW_EMPTY_REGISTRY
Append the following parameters to this line.
--ca-cert $PATH_TO_CA_CERT --enable-auth
For Docker Controller Setup
Use the mounted certificate folder that was configured in the server certificate stage. Copy the CA certificate to that folder on the host (just the certificate without the key).
Edit docker-compose.yml.
Search for the environment variables “CA_CERT” and “ENABLE_AUTH”. Edit them so they will look like the configuration below. (assuming that a certificate folder is already mounted at /mnt/cert).
anka-controller:
build:
context: .
dockerfile: anka-controller.docker
ports:
- "80:80"
# To change the port, change the above line: - "CUSTOM_PORT:80"
volumes:
# Path to ssl certificates directory
- /home/ubuntu/certs:/mnt/cert
depends_on:
- etcd
# - beanstalk
restart: always
environment:
USE_HTTPS: --use-https
# Server certificate pem
SERVER_CERT: --server-cert /mnt/cert/controller-crt.pem
# Server private key pem
SERVER_KEY: --server-key /mnt/cert/controller-key.pem
ENABLE_AUTH: --enable-auth
CA_CERT: --ca-cert /mnt/cert/ca-crt.pem
Testing your configuration for authentication
Restart your server. Your dashboard should not be working since authentication is now required. We can test the status endpoint with curl.
Execute the following command.
`curl -v “https://$HOST/api/v1/status” -k`
The response you should get is a ‘401 Authentication required’ response like the response below.
> GET /api/v1/status HTTP/2
> Host: localhost:8090
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 401
< content-type: application/json
< content-length: 54
< date: Thu, 28 Nov 2019 16:58:23 GMT
<
{"status":"FAIL","message":"Authentication Required"}
If this is the response you get, it means the authentication module is working.
Let’s try to get a response using the client certificate we created.
Execute the same command, but now pass client certificate and key (assuming those files are called ‘node-crt.pem’ and ‘node-key.pem’).
`curl -v “https://$HOST/api/v1/status” -k --cert node-crt.pem --key node-key.pem`
If everthing is configured correctly, you should see something like this.
> GET /api/v1/status HTTP/2
> Host: localhost:8090
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< cache-control: no-store
< content-type: application/json
< content-length: 188
< date: Thu, 28 Nov 2019 17:02:02 GMT
<
{"status":"OK","message":"","body":{"status":"Running","version":"1.3.1-13123799","registry_address":"https://10.0.0.3:8089","registry_status":"Running","license":"enterprise"}}
Joining Anka Client to Controller with client certificates
If you have any Anka nodes connected to your controller they now need the client certificate in order to work.
Copy the client certificate and key to the mac you are trying to join. Also copy the CA certificate used to create the controller certificates in case they are self signed (only copy the certificate, not the key).
Use ankacluster command to connect the agent to the controller in the following manner.
sudo ankacluster join https://$HOST --cert node-crt.pem --cert-key node-key.pem --cacert ca-crt.pem
You should see output similar to the following:
Testing connection to controller...: OK
Testing connection to registry….: OK
Configuring root token for the Controller Portal
Your nodes are working with client certificates and your api can be used with client certificates also. Now, the controller portal can be secured with the root token authentication option.
Your root token should be at least 10 notes long. For this example we will use 0987654321 but you can generate a strong token for securing your Controller Portal.
For Mac Controller
Edit /usr/local/bin/anka-controllerd.
The last line should be the controller’s run command and should look similar to this (it might have your certificates from before).
"$CONTROLLER_BIN" --listen_addr "$LISTEN_ADDRESS" --log_dir "$LOG_DIR" --data-dir "$DATA_DIR" $ANKA_REGISTRY $STANDALONE $ALLOW_EMPTY_REGISTRY
Append --root-token 0987654321 to this line.
For Docker Controller
Edit docker-compose.yml.
Search for the environment variable “ROOT_TOKEN”. Edit it so it will look like the configuration below.
anka-controller:
build:
context: .
dockerfile: anka-controller.docker
ports:
- "80:80"
# To change the port, change the above line: - "CUSTOM_PORT:80"
volumes:
# Path to ssl certificates directory
- /home/ubuntu/certs:/mnt/cert
depends_on:
- etcd
# - beanstalk
restart: always
environment:
USE_HTTPS: --use-https
# Server certificate pem
SERVER_CERT: --server-cert /mnt/cert/controller-crt.pem
# Server private key pem
SERVER_KEY: --server-key /mnt/cert/controller-key.pem
ENABLE_AUTH: --enable-auth
CA_CERT: --ca-cert /mnt/cert/ca-crt.pem
ROOT_TOKEN: --root-token 0987654321
Testing your configuration for root token based autbentication to the Controller Portal
Use the browser and go to your controller portal.
If everything is configured correctly, a login box should appear. Use the token string you created to login.