In September 2014, Cisco launched the UCS Mini for what they dubbed “Edge-Scale Computing”. The UCS Mini offering is a stripped down version of the classic UCS system with Fabric Interconnects. To learn about classic UCS, click here.
In this post I will go into some details about the UCS Mini that I’ve learned over the past few months. The information here will assume you have the basic knowledge about what the UCS Mini is and what components are involved. If you need a refresher, I’ve written up a small introduction here.
Let’s dive in.
Fabric Interconnect
The UCS FI 6324 is the same size as the UCS I/O Modules (IOM 2208XP, 2204XP, etc) and fits in the back IOM slot of the UCS Chassis 5108 (version 2).
The FI6324 has 4 fixed 10Gbps unified ports, a single 40Gbps QSFP+ port (which is a breakout to 4 x 10Gbps ports) a regular console port, management port (10/100/1000Mbps) and an USB port for offline firmware upgrades (only Cisco USB officially supported).
The fixed 10Gbps ports can be used as Server, Uplink and FC/FCoE Storage ports. The 40Gbps QSFP+ breakout port can only be used as Server, Appliance and FCoE Storage ports. Breakout scalability ports are named as Br-Eth1/5/1 to Br-Eth1/5/4:
When the system has been configured in a cluster, the two FI6324s will take over management services from each other. The subordinate will take over as primary if the primary FI6324 fails or when you hot swap it. The management and the vNics which are configured for fabric failover are failed over to the subordinate.
You can start with a single FI6324 when first deploying an UCS Mini and expand with a second FI6324 at a later date.
Scalability
In terms of scalability, the UCS Mini fits the “small branch” profile:
| Feature | Maximum |
|---|---|
| Switch Fabric Throughput | 500 Gbps |
| Ethernet port density | 4 Ports (1 / 10Gbps) |
| Fiberchannel port density | 4 Ports (4 / 8 Gbps) |
| MAC Address table size | 8000 |
| Number of VLANs | 200 |
| Number of IGMP Multicast Groups | 50 |
| Number of Virtual Interfaces (VIF) | 648 |
| Number of VIFs per VIC on blade servers | 116 |
| Number of VIFs per VIC on rack servers | 120 (* same as classic UCS) |
* Okay, so the amount of VLANs is not that high, but remember it’s for a branch location.
The UCS Mini currently supports up to 8 x B200 M3 blades and up to 7 rack servers (C220 M3 and C240 M3). In the future, the UCS Mini will most likely support attaching another chassis, expanding the support to 16 blade servers.
Features
When it comes to features, UCS Mini has some catching up to do. Most of the features you’d need in a branch office are available, some other classic UCS features are not yet ported to the UCS Mini software train.
| Supported | Not (yet) Supported |
|---|---|
| Network End-Host Mode | Network Switching Mode |
| FC Switching Mode | FC End-Host Mode |
| NIV | Port Security |
| VLAN | PVLANs |
| vNic Fabric Failover | FEX |
| QoS | NetFlow+ |
| Egress queuing | Ingress queuing |
| Ethernet SPAN | FC SPAN |
| FC/FCoE directly attached storage | |
| VMFEX for VMware and Hyper-V | |
| Only B200 M3 Blades |
Intra-connectivity
The UCS Fabric Interconnect 6100 and 6200 series use the L1 and L2 external ports (bond0) to form a cluster. You may have noticed that the FI6324 does not have these L1 and L2 ports, simply because there isn’t any space available. Among others, the cluster interconnect (eth2) has been moved to the backplane using direct traces.
The classic UCS platform uses SErial EPROM (SEEPROM) for split-brain avoidance. It uses all connected chassis to elect a primary in a split-brain situation. Whichever FI can claim the most chassis during this election gets to be the master. In the case of the UCS Mini, only the chassis which holds the FI6324s is used for this election, rack servers will not be used for this process.
Power enhancements
Due to the nature of the UCS Mini use cases (small branches, disaster recovery), the power management has been enhanced to fit more power restrained locations.
- Dual Voltage PSUs for high-line (220V) and low-line (110V) support.
- Dynamic Allocation: instead of allocating fixed power values to half or full width slots, UCS Manager takes available power, blade consumption and power priorities into account.
- Power Profiling: the minimum and maximum needed power are determined during blade discovery, using mprime in PNUOS for profiling. Can be disabled on a global level (Equipment -> Policies -> Global Policies -> Global Power Profiling Policy). When disabled, a minimum of 350W will be reserved.
- Reserved Power: 600W for an empty chassis, 25W for each slot (was 100W)
- Boot Staggering: UCS Manager looks to the available power to determine how many blades can be powered up at once. Before powering a blade, it’ll check the available power and stop the process if not enough is available.
- Budget Reclaim: When a blade is powered off, the power allocated to that blade will be returned to the power budget.
Firmware
Cisco currently has split off an infrastructure bundle train especially for the UCS Mini, starting with version 3.0. The classic UCS infrastructure bundle cannot be activated (it can be downloaded though) on the UCS Mini. The blade and rack server bundles and hardware catalog have remained the same, except for having the newer version.
- Infrastructure Bundle: ucs-mini-k9-bundle-infra.<version>.A.bin
- Contains: ucs-6324-k9-kickstart.<version>.bin, ucs-6324-k9-system.<version>.bin, ucs-manager-k9.<version>.bin
- Blade Server Bundle: ucs-k9-bundle-b-series.<version>.B.bin – same as classic Cisco UCS
- Rack Server Bundle: ucs-k9-bundle-c-series.<version>.C.bin – same as classic Cisco UCS
- Catalog: ucs-catalog.<version>.T.bin
Emulator
There is a new Cisco UCS Platform Emulator (UCS-PE 3.0(1cPE1)) version, which allows you to emulate the FI6324 and its unique features. Like the UCS firmware version 3.0, the UCS-PE 3.0 is a separate appliance for the time being. This new version only allows you to emulate the version 2 chassis (UCSB-5108-AC2 or UCSB-5108-DC2) and the FI6324 modules. UCS-PE 2.2 is still the emulator to use for classic UCS.
You can find the UCS Mini emulator here: https://communities.cisco.com/docs/DOC-53980
UCS Central
Cisco UCS Central is a virtual appliance you can deploy in your datacenter for a central location for UCS Management. You can add multiple separate UCS domains into UCS Central for central management, central resource pooling, standardisation of policies and simply ease of management of having a central entry point instead of several separate UCS domains to go to. With the release of UCS Mini, UCS Central has been updated to version 1.2 to support UCS Mini management.
New UCS Mini features (such as the scalability ports) are supported by UCS Central, but the workflows inside UCS Central remain the same.
UCS Central has been given a few WAN Optimising enhancements:
- Only the required firmware images are transferred to the UCS domain, eliminating excess bandwidth.
- New Network requirements: a minimum of 1.5Mbps bandwidth, a maximum latency of 500ms, jitter is no longer an issue.
- Restart-ability is supported for transfers that are cut off by intermittent WAN links.
- NFS is no longer used for file transfers to UCS Mini domains.
Thanks for sharing this, Martijn!
Great detailed Information. Thank you so much, Martijn 🙂
Excellent blog article, Martijn!
Thanks, Nice Information
In this blog, you have shared a complete information about UCS Mini. It looks a quite beneficial for the Cisci users.