ACS override patch nem működik, nem tudom szeparálni az IOMMU groupokat

 ( takipeti90 | 2019. február 22., péntek - 14:42 )

Sziasztok!

Az otthoni proxmoxos szerveremen létrehoztam egy VM-et a pfSense-nek és át akarom adni neki a külső PCI expresses hálókártyámat és egy M.2-es wifi adaptert. A probléma az, hogy az alaplapi hálókártya ugyanazon a PCI bridgen van, ezért egy IOMMU group-ba kerülnek, így viszont csak egyszerre mind a 3 eszközt tudom átadni a VM-nek. ACS override patch (pcie_acs_override=downstream,multifunction) sem csinál semmit.

Az alaplapom egy Asrock J3455-ITX, frissítettem a BIOS-t, engedélyeztem az intel virtualizációt és a VT-d is.


dmesg | grep -i iommu
[ 0.000000] Command line: BOOT_IMAGE=/ROOT/pve-1@/boot/vmlinuz-4.15.18-10-pve root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on pcie_acs_override=downstream,multifunction
[ 0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[ 0.000000] Kernel command line: BOOT_IMAGE=/ROOT/pve-1@/boot/vmlinuz-4.15.18-10-pve root=ZFS=rpool/ROOT/pve-1 ro root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on pcie_acs_override=downstream,multifunction
[ 0.000000] DMAR: IOMMU enabled
[ 0.004000] DMAR-IR: IOAPIC id 1 under DRHD base 0xfed65000 IOMMU 1
[ 1.329307] iommu: Adding device 0000:00:00.0 to group 0
[ 1.329322] iommu: Adding device 0000:00:02.0 to group 1
[ 1.329337] iommu: Adding device 0000:00:0e.0 to group 2
[ 1.329360] iommu: Adding device 0000:00:0f.0 to group 3
[ 1.329375] iommu: Adding device 0000:00:12.0 to group 4
[ 1.329429] iommu: Adding device 0000:00:13.0 to group 5
[ 1.329450] iommu: Adding device 0000:00:13.1 to group 5
[ 1.329469] iommu: Adding device 0000:00:13.2 to group 5
[ 1.329488] iommu: Adding device 0000:00:13.3 to group 5
[ 1.329511] iommu: Adding device 0000:00:15.0 to group 6
[ 1.329533] iommu: Adding device 0000:00:1f.0 to group 7
[ 1.329546] iommu: Adding device 0000:00:1f.1 to group 7
[ 1.329557] iommu: Adding device 0000:01:00.0 to group 5
[ 1.329569] iommu: Adding device 0000:02:00.0 to group 5
[ 1.329581] iommu: Adding device 0000:03:00.0 to group 5
[ 1.329596] iommu: Adding device 0000:04:00.0 to group 5

A külső hálózati kártya pedig ide van kötve:

lspci -tv
-[0000:00]-+-00.0 Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series Host Bridge
+-02.0 Intel Corporation Device 5a85
+-0e.0 Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series Audio Cluster
+-0f.0 Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series Trusted Execution Engine
+-12.0 Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series SATA AHCI Controller
+-13.0-[01]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
+-13.1-[02]----00.0 Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter
+-13.2-[03]----00.0 ASMedia Technology Inc. ASM1062 Serial ATA Controller
+-13.3-[04]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
+-15.0 Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series USB xHCI
+-1f.0 Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series Low Pin Count Interface
\-1f.1 Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series SMBus Controller

lspci -vvnnns 00:13.3
00:13.3 PCI bridge [0604]: Intel Corporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series PCI Express Port A #4 [8086:5adb] (rev fb) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0
ExtTag- RBE+
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 256 bytes, MaxReadReq 128 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
LnkCap: Port #6, Speed 5GT/s, Width x1, ASPM not supported, Exit Latency L0s <1us, L1 <4us
ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
Slot #5, PowerLimit 10.000W; Interlock- NoCompl+
SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
Changed: MRL- PresDet- LinkState+
RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
RootCap: CRSVisible-
RootSta: PME ReqID 0000, PMEStatus- PMEPending-
DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR+, OBFF Not Supported ARIFwd+
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled ARIFwd-
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
Address: fee002d8 Data: 0000
Capabilities: [90] Subsystem: ASRock Incorporation Atom/Celeron/Pentium Processor N4200/N3350/E3900 Series PCI Express Port A [1849:5adb]
Capabilities: [a0] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [100 v0] #00
Capabilities: [140 v1] Access Control Services
ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Capabilities: [150 v0] #00
Capabilities: [200 v1] L1 PM Substates
L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
PortCommonModeRestoreTime=40us PortTPowerOnTime=10us
L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
T_CommonMode=0us LTR1.2_Threshold=0ns
L1SubCtl2: T_PwrOn=10us
Kernel driver in use: pcieport
Kernel modules: shpchp

Esetleg van valakinek valami ötlete, hogy lehetne megoldani, hogy külön IOMMU groupba kerüljenek?