I hope to passthrough SAS HBA( LSI SAS3008 PCI-Express Fusion-MPT SAS-3) and HP410i raid controller to Proxmox VM.
RMRR issuse in PCIe passthrough
The SAS HBA fail to passthrough to Proxmox VM, and it shows following
error: 1
2
3vfio: failed to set iommu for container: Operation not permitted
dmesg | grep -e DMAR -e IOMMU
vfio-pci 0000:0X:YY.Z: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.1
2
3
4[ 58.606116] DMAR: DRHD: handling fault status reg 2
[ 58.606158] DMAR: [DMA Write] Request device [04:00.0] PASID ffffffff fault addr ffeee000 [fault reason 05] PTE Write access is not set
[ 102.466175] DMAR: DRHD: handling fault status reg 102
[ 102.466240] DMAR: [DMA Read] Request device [04:00.0] PASID ffffffff fault addr c763e000 [fault reason 06] PTE Read access is not set
HP ML350 G6 enables RMRR(VT-d Reserved Memory Reporting Regions) in
PCIe slot, but linux kernel disables RMRR feature for performance
issues. HP ML350 G6 can't disable RMRR by method(ref.1), so I patch
kernel following Feni's tutorial. After each kernel update, I have to
compile again. Aterfax publish
the precompiled packges, so I can use dpkg -i *.deb to
install the kernel. But you still can follow the latest tuturial to
compile kernel.
STEPS: 1. Download the build script (e.g. use
wget
https://raw.githubusercontent.com/kiler129/relax-intel-rmrr/master/build/proxmox/build.sh)
2. Run the build.sh script from terminal:
RMRR_AUTOINSTALL=1 bash ./build.sh 3. To activate patch,
add intel_iommu=relax_rmrr to Linux boot
args(/etc/default/grub), and run update-grub 4. After
reboot, uname -r should show a version ending with
-pve-relaxablermrr. And
dmesg | grep 'Intel-IOMMU' show similar result.
1 | root@hp6:~# dmesg | grep 'Intel-IOMMU' |
Use P410i as HBA card
To turn hp P410i to HBA model and passthrough it, I follow the hpsahba tool.
1 | root@hp6:~# git clone https://github.com/im-0/hpsahba.git |
I don't known why the hpsa-dkms doesn't generate hpsa module with
hpsa_use_nvram_hba_flag during kernel upgrade. If I remove
hpsa-dkms by dkms remove hpsa-dkms/1.0 --all and patch the
kernel again, it works. I can build binary files, but no idea how to
package as rpm.
I try to patch P410i in the opensuse VM. https://github.com/openSUSE/kernel-source/blob/master/doc/README.SUSE https://forums.opensuse.org/showthread.php/569506-Kernel-Update https://forum.proxmox.com/threads/hp-proliant-microserver-gen8-raidcontroller-hp-p410-passthrough-probleme.30547/#post-153784
Install kernel-source. Change to the /usr/src/linux directory.
Create a build directory for use in configuring and building the kernel. Using /usr/src/linux directly requires root priviledges and will cause problems if you need to build kernel modules for other installed kernels.
Configure the kernel (for example, ``make -C /usr/src/linux O=$(pwd) oldconfig'', see HOW TO CONFIGURE THE KERNEL SOURCES).
Build the kernel and all its modules (``make'').
Make sure that /etc/modprobe.d/unsupported-modules contains
allow_unsupported_modules 1otherwise modprobe will refuse to load any modules.
Install the kernel and the modules (
make modules_install'', followed bymake install''). This will automatically create an initrd for the new kernel as well (see ``mkinitrd -h'').
1 | [ 103.609595] DMAR: DRHD: handling fault status reg 2 |
1 | dmesg | grep -e DMAR -e IOMMU |
Fail https://www.reddit.com/r/Proxmox/comments/hhx77k/the_importance_of_iommupt_with_gpu_pass_through_i/ https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/html/installation_guide/appe-configuring_a_hypervisor_host_for_pci_passthrough
https://forum.proxmox.com/threads/boot-failure-on-upgrade-to-ve-7-2-with-kernel-5-15-35-1-pve.109144/page-2 This one doesn't work for me.
Remote access by ILO2
Because even on the latest firmware (2.33 as of 2021-04-16), due to hardware limitations, iLO2 does not support modern TLS (and ciphers). But recently java disable TLS 1.0 and 1.1.(ref 8) Therefore, adjusting the JRE's security settings is necessary. Seemingly, this can not be done at runtime, so a custom security file has to be passed to Java.(ref.7)
This viewer doesn't run on my openSUSE. Even I follow tutorial in
ref.9 to modify the system java.security, I still get error
of SSLException: No appropriate protocol. I doubt that java
interacts with java.security in wrong way. But, I'm lucky
to find that it works well in RPI 3B. 1
2
3git clone https://github.com/scrapes/ILO2-Standalone-Remote-Console.git
./gradlew build
./java -Djava.security.properties=java.security -jar ILO2RemCon.jar -c config.properties
Ref:
- https://www.jimmdenton.com/proliant-intel-dpdk/