Find the answer to your Linux question:
Results 1 to 4 of 4
I'm running a 2.6.30.9 kernel on a SuperMicro/Intel chipset, and I'm writing a module for a PCIe device that uses MSI. The first time I load the module using insmod, ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Apr 2010
    Posts
    1

    Reloading Module Stops MSI Interrupts


    I'm running a 2.6.30.9 kernel on a SuperMicro/Intel chipset, and I'm writing a module for a PCIe device that uses MSI. The first time I load the module using insmod, interrupts work correctly, and the ISR is run as it should. When I do a rmmod and another insmod, interrupts are not caught by my module.

    I've included the code from the init and exit functions below, but I have followed the directions in the MSI HOW-TO.txt file to call pci_enable_msi then request_irq in the init function. In the exit function, I call free_irq, then pci_disable_msi.

    When interrupts are working (first insmod after reboot), /proc/interrupts shows the following after running a test program:

    Code:
    53:      33289     566723          0          0          0          0          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      RMA Interrupt
    When interrupts aren't working after reloading the module, /proc/interrupts show the following after running the same test program:

    Code:
    53:      0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      RMA Interrupt
    I know that the problem isn't with the device because a reboot will fix the problem without restarting the device. Additionally, with a signal tap on the FPGA, the IRQ request signal remains high for the time when the first interrupt should come. The acknowledge signal isn't coming back to the device because the module isn't receiving the interrupts and going through the ISR.

    Both when interrupts are working, and when they aren't, lspci -vv show the following:

    Code:
    05:00.0 Class ff00: Altera Corporation Unknown device 0004 (rev 01)
            Subsystem: Altera Corporation Unknown device 0004
            Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
            Latency: 0, Cache Line Size: 64 bytes
            Interrupt: pin A routed to IRQ 53
            Region 0: Memory at 680000000 (64-bit, prefetchable) [size=1024M]
            Capabilities: [50] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
                    Address: 00000000fee00000  Data: 40ba
            Capabilities: [68] MSI-X: Enable- Mask- TabSize=2
                    Vector table: BAR=0 offset=00000400
                    PBA: BAR=0 offset=00000500
            Capabilities: [78] Power Management version 3
                    Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                    Status: D0 PME-Enable- DSel=0 DScale=0 PME-
            Capabilities: [80] Express Endpoint IRQ 0
                    Device: Supported: MaxPayload 2048 bytes, PhantFunc 0, ExtTag-
                    Device: Latency L0s <64ns, L1 <1us
                    Device: AtnBtn- AtnInd- PwrInd-
                    Device: Errors: Correctable- Non-Fatal- Fatal- Unsupported-
                    Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                    Device: MaxPayload 256 bytes, MaxReadReq 512 bytes
                    Link: Supported Speed unknown, Width x4, ASPM L0s, Port 1
                    Link: Latency L0s unlimited, L1 unlimited
                    Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
                    Link: Speed unknown, Width x4
    Here is my source code for the init and exit functions for loading and unloading the module:

    Code:
    static int __init ssrma_init(void){
      int retval;
      ssrma_DEV_T *rma_dev;
      irq_handler_t irq_handler = &ssrma_pciIsr;
      rma_dev = &install_device;
    
      /* Register the driver with the PCI core */
      retval = pci_register_driver(&pci_driver);
      if(retval) return retval;
    
      /* Setup the cdev portion of the rma_device struct */
      retval = rma_setup_cdev(rma_dev);
      if(retval){
        goto registered;
      }
    
      /* Verify the device is found on a bus */
      rma_dev->pci_dev = pci_get_device(ssrma_VENDOR_ID, ssrma_DEVICE_ID, NULL);
      if(!rma_dev->pci_dev){
            retval = -ENODEV; // No such device
        goto cleanup;
      }
      /* Add the device to the device structure */
      pci_dev_put(rma_dev->pci_dev);
    
      /* Enable the MSI interrupts */
      if(pci_enable_msi(rma_dev->pci_dev)){
        retval = -1;
        goto cleanup;
      }
     /* Setup the interrupt ISR */
      if(request_irq(rma_dev->pci_dev->irq, irq_handler, IRQF_DISABLED, "RMA Interrupt", rma_dev)){
           retval = -1;
        goto msi_cleanup;
      }
    
      return 0;
    
      ioremap_cleanup:
        /* Unmap the mapped memory space */
        iounmap(rma_dev->virt_start_addr);
        /* Free the requested IRQ */
        free_irq(rma_dev->pci_dev->irq, rma_dev);
      
    msi_cleanup:
        /* Disable the MSI interrupts */
        pci_disable_msi(rma_dev->pci_dev);
      
    cleanup:
        /* Remove the device */
        cdev_del(&rma_dev->cdev);
     
     registered:
        /* Unregister the driver */
        pci_unregister_driver(&pci_driver);
      return retval;
    } /* end rma_init */
    
    
    
    static void __exit ssrma_exit(void){
      ssrma_DEV_T *rma_dev;
      rma_dev = &install_device;
    
      /* Unmap the mapped space */
      iounmap(rma_dev->virt_start_addr);
    
      /* Free the requested IRQ */
      free_irq(rma_dev->pci_dev->irq, rma_dev);
    
      /* Disable the MSI interrupts */
      pci_disable_msi(rma_dev->pci_dev);
    
      /* Remove the device */
      cdev_del(&rma_dev->cdev);
    
      /* Unregister the driver */
      pci_unregister_driver(&pci_driver);
    
    } /* end rma_exit */

    If you have had a similar problem, or know what I'm doing wrong, please let me know. If you need more information, I would be happy to provide it. It is frustrating to have to reboot everytime I change the module code. Thank you.

  2. #2
    ths
    ths is offline
    Just Joined!
    Join Date
    Jun 2010
    Posts
    2
    Hi,

    I have the same problem here that I don't get any MSI interrupts after reloading my driver module.
    Did you find any solution for this problem or anyone else?

    Thank you

  3. #3
    ths
    ths is offline
    Just Joined!
    Join Date
    Jun 2010
    Posts
    2
    I found my problem here:
    Calling pci_disable_device removes the busmaster flag from the PCI device.
    But pci_enable_device doesn't set it again when loading the module.
    And for MSI the busmaster flag is needed because the interrupt is a data package
    over the data bus, not an IRQ line.

    ths

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined!
    Join Date
    Mar 2012
    Posts
    1
    Hi,

    I am on my way to implement MSI:
    On the Hardware side all it's ok but now I am looking for a guidance (piece of code)
    on testing MSI Capabilities on the Software side(Driver)
    For that I would appreciate any help to move forward

    Regards,

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •