This guide is based on the NI forums thread Provide a Virtual Machine (VM) in which to run LV RT systems on and is intended to be a brief summery.
NI provides a Linux RT Recovery-Image in the folder
C:\Program Files (x86)\National Instruments\RT Images\Utilities\Linux RT PXI Safemode\8.5\standard_x64_recovery.iso (Version 8.5 in this case).
If the image is not there, you can install the package
NI Linux RT System Image with the NI Package Manager.
In VMware Workstation (or Fusion) create a new Virtual Machine with the following configuration.
After the VM is created and before you start it, you have to go into the VM Settings ⇾ Options Tab ⇾ Advanced and set the Firmware type to UEFI.
On first boot there will be a screen “NI Linux Real-Time Boot Options”. Press “e” to edit the Grub boot commands like in the screenshot below. With this command the operating system will be installed without further requests. Press “F10” to start the installation.