Foreman – PXEless Bare Metal & Virtual Machine Provisioning

Foreman is an open source package that provisions bare metal and virtual machine in public and private clouds. For provisioning, Foreman is able to do PXE-less and PXE installations of Bare Metal & VM servers. When doing PXE-less provisioning the bootdisk or discovery plugins are necessary. For PXE provisioning, Foreman makes use of PXE, DHCP, DNS, TFTP, and kickstart (CentOS & RHEL). After servers are provisioned, Foreman is able to configure servers using Puppet. Chef, Salt, Ansible, and Bash are also supported. Puppet is the default configuration tool installed with the Foreman server.

Presented is pertinent information and steps required to provision bare metal and virtual machines using a PXE-less method. This post assumes Foreman 1.1.9 is installed on CentOS 7 / RHEL 7 using Puppet5.

If you have not yet installed foreman, visit our post “Foreman – Install and Configure on CentOS / RHEL 7.x

Foreman uses a plugin model to extend function outside of Foreman core. Available plugins can be reviewed at

PXE-less OS Installation on Base Metal or VM Servers (what some call “MaaS – Metal as a Service”)

To do PXE-less OS installations of Bare Metal or VM servers requires either the Foreman Discovery or BootDisk plugin to be installed on the Foreman server.

NOTE – The BootDisk plugin requires a network interface card (NIC) that support iPXE which is less commonly available with older NIC adapters. To avoid PXE-less boot issues with servers, this post will focus attention on the Foreman Discovery plugin. An evaluation of the Foreman BootDisk plugin will be completed in the future to better understand the scope of this limit.

NOTE – At this time, the Discovery plugin for each Foreman release is available on the foreman website within the plugins directory for a given Foreman release.

How the Foreman Discovery plugin can be used to provision a server depends on the state of

  1. The boot settings of the Bare Metal or VM server (PXE or iPXE boot order)
  2. The protocols and services available on the network (DHCP, DNS, and TFTP)
  3. The auto-provisioning rules that have been created in Foreman.

PXE-less host discovery requires the Foreman Discover Image (a bootable ISO image) be placed on a CD, DVD or USB stick to boot the server to be discovered by Foreman and start the installation process of the desired OS image via Foreman. Using this process, the automatic discovery process is not started automatically as with PXE. A user must select the primary network interface interactively. The user may need to configure network credentials and URL of the foreman server.

The Foreman Discovery Image sends facts via kexec that signals the Discovery plugin to initiate kexec reload instead of a reboot. When the host is provisioned either manually or via discovery rule(s), the instance loads the OS installer via the kexec utility.

Before a system can be discovered without PXE, there must be a discovery rule associated with a hostgroup having an OS associated with a “kexec” template. This template is used to pass in parameters for the kexec call (kernel, initrd, append line). The Discovery plugin provides templates for Red Hat distributions.

Ensure the associated provisioning template is configured with static networking if the deployment does not provide DHCP services. Otherwise, Anaconda will fail to install the system after kernel reload. Also, verify netmask, gateway, and primary DNS server is configured on the associated Subnet.

Unattended and semi-automatic mode

All interactive screens can be fully automated via kernel command line that can be either entered during SYSLINUX/Grub2 boot screen or burnt into the image (see below). Options details follow:

proxy.url=https://foreman server:443


When px.ip and/or are omitted, the Discovery Plugin Image tries to acquire network credentials over DHCP, which is useful in PXE-less environments with DHCP server deployed. Therefore the absolute minimum set of options in this case is:

proxy.url=https://foreman server:443

All values must be confirmed in the text interface, which gives semi-automatic behavior. To initiate the provisioning without questions (unattended mode or fully automatic), add the following option:


When fdi.pxauto is not provided, Ok/Next buttons receive focused to navigate the interface fast.

The pxmac option defines the provisioning interface to be used to acquire network credentials (optionally) and send facts. If omitted in unattended mode, the first NIC with a link is picked up (in alphabetical order by network identifier in case of multiple items). In the semi-automated mode, the screen will appear to select the correct interface.

Remastering Discovery Image

Unattended and semi-automatic mode described above requires some kernel command line options to be present. To avoid typing them in PXELinux/Grub2 loaders, it is possible to re-master the discovery image with the desired option in the image.

The helper script discovery_remaster is shipped with the foreman_discovery plugin. It can be used to make a copy of the ISO image with additional kernel command line options. The following command creates a copy in the same directory with date/time stamp in the name –

sudo discovery-remaster fdi-bootable-3.4.X.iso "fdi.pxip= fdi.pxgw= fdi.pxdns= proxy.url= proxy.type=proxy fdi.pxfactname1=myfact fdi.pxfactvalue1=somevalue fdi.pxmac=52:54:00:be:8e:8c fdi.pxauto=1"

Provisioning without PXE/DHCP

  • Setup auto-provisioning rules (optional)
  • Transfer the Discovery ISO image onto a USB stick or CDROM and boot it.
  • Select Manual network setup
  • Select primary (provisioning) interface if the host has multiple NICs
  • Enter IPv4 credentials
  • The host is discovered
  • The host reloads kernel into installer either manually or via auto-provisioning

Provisioning with DHCP (no PXE)

  • Transfer the Discovery ISO image onto a USB stick or CDROM and boot it.
  • Select Discover with DHCP
  • Select primary (provisioning) interface if the host has multiple NICs

Semi-automated workflow w/o PXE/DHCP

  • Modify the Discovery ISO with the following options:
    • fdi.pxgw=
    • fdi.pxdns=
    • proxy.url=https://foreman:443
    • proxy.type=foreman
    • fdi.pxfactname1=deployment_type
    • fdi.pxfactvalue1=database_server
  • Transfer the Discovery ISO image onto a USB stick or CDROM and boot it.
  • Select primary (provisioning) interface if the host has multiple NICs
  • Enter IPv4 credentials (gateway and DNS is already populated)
  • Select primary (provisioning) interface

Fully automated workflow w/o PXE/DHCP

  • Modify the Discovery ISO and set all required options including MAC and IP address
  • Set fdi.pxauto option to 1
  • Transfer the ISO image onto a USB stick or CDROM and boot it.
  • Discovered node automatically uploads facts and reloads kernel into the installer

Step:1 Plugin Installation

There are four different components to install: Foreman Discovery plugin, Proxy Discovery plugin (Smart Proxy Discovery), Foreman CLI (Hammer) plugin and Foreman Discovery Image. At this time, we will not be using the Proxy Discovery plugin.

The foreman-installer is able to install plugins. To do this, re-execute the installer with the following option:

# foreman-installer --enable-foreman-plugin-discovery

Step:2 Download the Foreman Discovery Image

The Foreman Discovery Image is a small CentOS 7 Boot image with modifications to provide a text menu that allows collection of minimal information required for PXE-less server booting and Installation of the desired OS image.

To download the Forman Discovery Image and place it on our tftp server –

# wget
-O - | tar x --overwrite -C /var/lib/tftpboot/boot

Creating a custom Foreman Discover Image

For information on creating a custom Foreman Discovery Image, visit the foreman-discovery-image git repository and review the README file for instructions

Extending the downloaded image

It is possible to extend the Foreman Discovery Image with custom facts, software or device drivers easily. There are two ways of doing that.

1) Runtime extensions

Provide a zip file containing extra code for the image to use. First, create a directory structure like:

├── autostart.d
│   └──
├── bin
│   └── ntpdate
├── facts
│   └── test.rb
└── lib
    └── ruby
        └── test.rb

autostart.d contains scripts that will be executed in POSIX order by the image as it starts up before the host is registered to Foreman. bin is added to PATH, you can place binaries here and use them in the autostart scripts. facts is added to FACTERLIB so that custom facts may be configured and sent to Foreman. lib is added to LD_LIBRARY_PATH and lib/ruby is added to RUBYLIB, so that binaries in bin are executed properly.

Environment variables (PATH, LD_LIBRARY_PATH, RUBYLIB, and FACTERLIB) are appended to. If you need to specify the path to something explicitly in your scripts, the zip contents are extracted to /opt/extension on the image.

An example structure is provided in example_zip within the repo. You can zip up your structure with zip -r .

You can create multiple zip files, but be aware they will be extracted to the same place on the discovery image, so files in later zips will overwrite earlier ones if they have the same filename.

To inform the image of the extensions it should use, place your zip(s) on your TFTP server along with the discovery image, and then update your PXELinux APPEND line with fdi.zips=<path-to-zip>,<path-to-zip>, where the paths are relative to the TFTP root. So if you have two zips at $TFTP/ and $TFTP/boot/, you would use,boot/

2) Image building with extensions

The same ZIP file with the structure described above can be injected into a custom image. This is useful for network drivers when there is a chicken and egg problem.

To build a discovery image with extensions, please visit the foreman-discovery-image git repository and find the README for further instructions.

Step:3 Install the Foreman CLI (Hammer) plugin

The hammer plugin should be included with the Foreman installation. To install the hammer plugin –

# yum install tfm-rubygem-hammer_cli_foreman_discovery

The plugin must be enabled in /etc/hammer/cli.modules.d/foreman_discovery.yml as follows:

    :enable_module: true

Validate the Discovery plugin using Hammer hammer -h and check that the discovery command is listed.

$ hammer -h
 discovery                     Discovery related actions.

The actions you can use with discovery will appear as follows:

$ hammer discovery -h
    hammer discovery [OPTIONS] SUBCOMMAND [ARG] ...

 SUBCOMMAND                    subcommand
 [ARG] ...                     subcommand arguments

 auto-provision                Auto provision a host
 delete                        Delete a discovered host
 facts                         Show a discovered host
 info                          Show a discovered host
 list                          List all discovered hosts
 provision                     Provision a discovered host
 reboot                        Reboot a host
 refresh-facts                 Refresh the facts of a host

 -h, --help                    print help

For example to reboot a discovered_host:

$ hammer discovery reboot -h
    hammer discovery reboot [OPTIONS]

 --id ID
 --name NAME                   Name to search by
 -h, --help                    print help

$ hammer discovery reboot --id 130
Host reboot started

Step:4 Foreman Discovery Plugin Configuration

Foreman Discovery relies on intercepting the normal boot process for servers not yet registered in Foreman. To achieve this, the PXE default.cfg file needs to be altered to instruct new machines to boot the discovery image.

Foreman “Default PXE” template

Select Hosts->Provisioning Templates, edit PXELinux global default template to change the default entry from localboot to discovery –

ONTIMEOUT discovery

The template contains a menu entry rendered from pxelinux_discovery snippet –

LABEL discovery
MENU LABEL Foreman Discovery Image
KERNEL boot/fdi-image/vmlinuz0
APPEND initrd=boot/fdi-image/initrd0.img ... proxy.url=<%= foreman_server_url %> proxy.type=foreman

The proxy.type option can be either proxy or foreman. If set to proxy, all communication goes through Smart Proxy. If set to foreman, all communication goes directly to Foreman (legacy mode). This is the default when not specified.

The proxy.url specifies URL of the Smart Proxy or Foreman depending on the previous setting. For backward compatibility, foreman.url is an alias for this setting.

If direct connection is not possible, enable Smart Proxy Discovery plugin (see above) to forward communication and configure the APPEND line in the following way:

APPEND ... proxy.url=https://FOREMAN_PROXY_INSTANCE:8443 proxy.type=proxy

Alternatively, the Discovery image also searches for DNS SRV record named _x-foreman._tcp. If the DNS server is configured for this (see the example for ISC BIND below), then it is not required to provide foreman.url at all, but proxy.type is still required if the SRV record is configured to point to a proxy.

Here is an example of ISC BIND configuration statement to set foreman A record to be used with HTTPS protocol.

_x-foreman._tcp SRV 0 5 443 foreman

You can mix both approaches and override SRV record with the command line.

It is important to keep IPAPPEND 2 option which is a key configuration option to detect interface connected to provisioning network. Also never change or remove root option, otherwise the image will not boot properly.

It is important to know that DNS servers from DHCP are taken into account for only for the interface that was specified via the BOOTIF option. This means when a system has multiple NICs, DNS will work for the correct interface – the one that was booted from.

Once the template has been updated, select Build PXE Default button (at the top of the Provisioning Templates page). This will instruct TFTP to rewrite the pxelinux.cfg/default file.

Note – Repeat this step every time a change is made to the default template

Using Organizations and Locations

If Locations and/or Organizations are enabled, Foreman assigns Organization and Location to discovered hosts according to the following rules from top to bottom:

  • According to the discovery_organization or discovery_location, if present. These can be set under Administer > Settings > Discovered.
  • If foreman_organization or foreman_location is present, set accordingly. Fact names which are looked up can be configured in Administer > Settings > Puppet section as Organization/Location fact setting.
  • If Subnet was determined for a particular discovered host, use the first Organization and Location associated with the Subnet.
  • Select the first Organization / Location ordered by title (name the org/loc was created with).

Organization or Location can be changed via the “bulk actions” menu which appears once more discovered hosts are selected.

Global Settings

  • Interface fact – Fact name to use for primary interface detection. It specifies which incoming fact should be used to get the MAC address. By default, the PXELinux BOOTIF kernel command line option is used which returns the MAC address of the interface used for boot.
  • Auto-provisioning – Automatically provision newly discovered hosts, according to the provisioning rules. Note it is set to false by default. If you want to trigger provisioning automatically with rules, you need to turn this setting on. It’s recommended to try provision manually first to test before proceeding.
  • Fact columns – Extra factor columns to show in host lists. It is possible to add any fact reported by Factor via this global option onto the Discovered Hosts page table as a new column. To do that, set the value of this setting to the name of a fact reported. To hide the new column, set to a blank value. To show multiple columns, separate fact names by a comma.
  • Clean all facts – Clean all reported facts during provisioning (except discovery facts).
  • Hostname facts – List of facts to use for the hostname (separated by a comma, first wins).
  • Reboot – Automatically reboot or kexec discovered host during provisioning.
  • Hostname prefix – The default prefix to use for the hostname, must start with a letter.
  • Highlighted facts – Regex to organize facts for highlights section – e.g. ^(abc|cde)$.
  • Storage facts – Regex to organize facts for storage section.
  • Software facts – Regex to organize facts for the software section.
  • Hardware facts – Regex to organize facts for hardware section.
  • Network facts – Regex to organize facts for network section.
  • IPMI facts – Regex to organize facts for ipmi section.
  • Lock PXE – Automatically generate PXE configuration to pin a newly discovered host to discovery.
  • Locked template name – PXE template to be used when pinning a host to discovery. Each individual PXE template (PXELinux, PXEGrub, PXEGrub2) has its own entry.
  • Force DNS – DNS entry will be created when provisioning discovered host.
Discovery image kernel options

Foreman discovery image recognizes the following list of configuration options:

  • proxy.url – URL to Foreman or Proxy (if omitted DNS SRV lookup is made)
  • proxy.type – endpoint type: “foreman” or “proxy”
  • fdi.zips – extensions to download
  • fdi.zipserver – override TFTP server reported by DHCP
  • fdi.initnet=all/bootif – initialize all or pxe NICs (default) during startup
  • fdi.ssh – configure ssh daemon after start (1 to enable)
  • fdi.rootpw – configure ssh daemon password (plain string)
  • fdi.countdown – initial countdown in seconds before registration attempt (45 by default)
  • fdi.uploadsleep – seconds between factor runs (30 by default)
  • fdi.cachefacts – the number of fact uploads without caching (0 by default)
  • fdi.px* – PXE-less workflow (described below)
  • fdi.dhcp_timeout – DHCP NetworkManager timeout in seconds (300 by default)
  • fdi.ipwait – wait time for IP to be available in proxy SSL cert start (120 by default)
  • fdi.nmwait – nmcli –wait option for NetworkManager (120 by default)
  • fdi.dns_nameserver – nameserver to use for DNS SRV record
  • fdi.proxy_cert_days – number of days HTTPS self-signed cert is valid (999 by default)
  • fdi.dns_search – search domain to use for DNS SRV record
  • fdi.dns_ndots – ndots option to use for DNS SRV record
  • fdi.vlan.primary – VLAN ID to set for the primary interface

By default, the image tries to send initial facts multiple times. The delay between factor runs can be changed with fdi.uploadsleep. Once facts are successfully sent to Foreman server, additional uploads are done only on hardware configuration change (e.g. disk drive is added). It is possible to force an arbitrary amount of uploads via the optionfdi.cachefacts. A reasonable value is 2-3 and also increase tofdi.uploadsleep more than 60 seconds to prevent race conditions when provisioning hosts (host could be discovered the second time when provisioning window is hit)


The plugin will create a Role called Discovery when first started. This can be assigned to roles for non-admins to allow them to use the discovery plugin. Alternatively, assign the permissionperform_discovery to an existing Role

Usage of Foreman Discover plugin

If a server is booted on the foreman provisioning network and set to boot using PXE, it will register with Foreman and appear in Hosts -> Discovered Hosts

Manual Provisioning of servers

Select a Discovered Host and select Provision.

This will redirect to the normal Edit page for a Host, with the discovered data filled in where possible. Fill in any additional information.

Select Save

Foreman will modify hosts PXELinux file on the TFTP server and reboot the discovered host.

The host will boot the install image of the chosen OS, and finally into the installed OS.

NOTE – To re-install an OS, delete a host and reboot to Discover and Re-Provision.

Automatic Provisioning of servers

Using predefine provisioning rules, it is possible to trigger desired behaviors. Select Configure -> Discovery rules and create a new rule:

  • Name represents rule name shown to the users. It must not contain spaces or non-alphanumeric characters.
  • Search statement is used to match discovered hosts for the particular rule. Use Scoped Search syntax to define it. Examples are shown below.
  • Host Group is assigned to a matching host before starting provisioning process. It is very important the selected Host Group has all the required parameters set (domain, subnet, root password), otherwise provisioning process will fail.
  • Hostname defines a pattern to assign human-readable hostnames to the matching hosts. When left empty, hostname is assigned in a form of macMACADDRESS by default. The same syntax as for provisioning templates is used. See below for more information and examples.
  • Hosts limit enables to limit maximum amount of provisioned hosts per rule. If a limit was reached, the rule will not take effect until one or more hosts are deleted. Typical use case are rules per server rack or row when it is necessary to change provisioning parameters like hostname or host group per each entry.
  • Priority specifies the order in which rules are applied. Must be greater than zero. Rules with lower priority number win. Rule that was created first wins in case of equal priority.
  • Enabled flag is used for a temporary shutdown of rules.
  • OrganizationsLocations discovered hosts can only auto provision from rules in the same organization/location as the discovered host. Host group taxonomy is enforced, therefore it is not possible to provision into a host group that is not associated with the discovery rule.

Once rules are defined, the good practice is to discover a host and apply the rules using Auto discover button on the host. By default, Foreman does not trigger autodiscovery automatically. This must be explicitly turned on in Administer > Settings > Discovered > discovery_auto.

Search syntax

An easy method to test rules is via search patterns in Discovered hosts list. Search returns the same results that will be processed by rules. Typical search fields are facts, they all start with “facts.”. Auto completion can be used to browse the facts as well as discovered hosts detail screen. Typical search queries:

facts.architecture = x86_64
facts.bios_vendor ~ 'Dell*'
facts.macaddress = "aa:bb:cc:dd:ee:ff"
facts.macaddress_eth0 = "aa:bb:cc:dd:ee:ff"
facts.ipaddress_eth1 ~ "192.168.*"
facts.discovery_proxy_uri = "https://my_proxy:8443"

All facts are simple strings, it is not possible to do numeric comparisons.

Some important facts are extracted and converted to numbers –

  • cpu_count – number of CPUs
  • disk_count – number of disks attached
  • disks_size – total amount of disk space (in MiB)

Possible queries are:

cpu_count >= 8
disk_count < 10
disks_size > 1000000

See the searching section for more information.

Hostname patterns

The target hostname template pattern has the same syntax as in Provisioning Templates (ERB). Domains are appended automatically. A hostname based on MAC address will be used when left blank. In addition to @host attribute function rand for random integers is available.


application-server-<%= rand(99999) %>
load-balancer-<%= @host.facts['bios_vendor'] + '-' + rand(99999).to_s %>
wwwsrv-<%= %>
minion-<%= %>
db-server-<%= @host.ip.gsub('.','-') + '-' + %>

When creating hostname patterns, make sure the resulting host names are unique. Hostnames must not start with numbers. A good approach is to use unique information provided by a factor (MAC address, BIOS or serial ID) or to randomize the hostname

Step:5 Create the OS installation media to be deployed on bare metal or VM servers

In our case, we have already created a yum server, and ftp server with CentOS 7 / RHEL 7 ISO images.

The CentOS 7 ISO file is mounted at “/var/ftp/pub/CentOS_7_x86_64

Login to Foreman and navigate to Hosts –> Installation media and select New Medium

Name of Installation Media : CentOS_x86_64

Path : ftp://<serverIP>/pub/CentOS_7_x86_64/

Choose Family as “Red Hat”


Step:6 Provide operating system and deployment protocol information to Foreman

navigate to Hosts –> Operating systems and then select New Operating system

Name (of the OS): CentOS

Major version: 7

Minor version: 4

Description: CentOS 7.4

Family: Redhat

Root password hash: SHA256

Architectures: x86_64

Select the Partition Table Tab, then select Kickstart default in the listbox

Select the Installation media Tab, then select CentOS_7_x86_64 in the listbox


To define the Templates for our new Operating system, navigate to Hosts –> Provisioning Templates

We will select 5 templates, then associate these templates with our new Operating system CentOS 7.2

The first “Provisioning template” to select is Kickstart default and associate the template to our operating system CentOS 7.2 then Submit. Do the same with Kickstart default finish, Kickstart default iPXE, Kickstart default PXELinux, and Kickstart default user data.

Select Hosts –> Operating system, and select CentOS 7.2 to associate the templates from Associate tab


Step:7 Provide networking information to Foreman

Select the Infrastructure Tab -> Provisioning setup, then select the Foreman server that will be used for provisioning

Specify the subnet name and starting / ending ip addresses of the subnet

Name: cluster X addresses

Network address:

Start IP range:

End IP range:


Copy the foreman-installer command line from the Install provisioning with DHCP and paste it to a shell window on the foreman server.

# foreman-installer

Once the foreman-installer completes, select Next

Select the Installation Media Tab, then select existing media CentOS_7_x86_64


Step:8 (optional) Edit Provisioning Template (Kickstart Default) to download puppet from local repo

This step is optional. The default provisioning template assumes servers have Internet connectivity. This optional step shows how to modify the templet to use an internal repo server. Let’s assume puppet packages have been placed on our ftp server at /var/ftp/pub/puppet_rpms/

Select Hosts -> Provisioning Templates, then select Kickstart default

replace the puppet code:

repo –name=”EPEL”

with the puppet code:

repo --name=”puppet-pkg” --baseurl=

Also remove the epel-release package from the package list

Step:9 Create a New Host in Foreman that will be Provisioned

Select Hosts Tab –> New Host

Specify the name, Host Group, and other required parameters.

Name: bigaperf001

Deploy On: Bare Metal

NOTE – the Puppet Classes Tab is used to define puppet classes or modules that will be executed after the OS installation. This can allow additional software, configuration or tuning to be done after OS installation.

Select the Interface Tab to define ethernet interface parameters.

Specify the MAC address of the network interface on the server to be provisioned through foreman. The network interface name (such as enp0s3) and IP address will be gathered automatically from the provision subnet.


Select Operating System Tab

Select Operating system CentOS 7.2

Media CentOS_7_x86_64

Partition Table Kickstart default

Root password to be used during OS installation


The new Host build status will become Pending Installation

NOTE – New Hosts that will be provisioned in the same subnet, OS level, kickstart file, and deployment protocol can be added rapidly by starting at step#9.

Step:10 Provisioning bare metal or VM via PXE-less

Boot the server with the Discovery Image loaded and configured at the top of the boot order

The bare metal or VM will be assigned an IP address provided by the Discovery Image

OS installation will be performed in text mode following the kickstart script provided by Foreman

Once OS installation is finished, Foreman will reboot the server