Author Topic: IPL cycle failure  (Read 1186 times)

Borley

  • Full Member
  • ***
  • Posts: 178
  • Karma: +16/-0
    • View Profile
IPL cycle failure
« on: October 22, 2024, 01:03:08 pm »
I have recently been unable to boot one of my Blackbirds (Rev 1.02). This thread will serve as notes for my troubleshoot.

Problem: The board BMC starts when mains power is available and the bottom left LED (D7000) flashes rapidly for 2-3 seconds before entering into a slower pulse indicating BMC bringup. However, upon starting chassis power, the board immediately restarts back to BMC bringup. System fans just barely begin to turn before it resets like this.

I can log into the BMC without issue and it reports (possibly old) critical failures:

Code: [Select]
HIGH Error: xyz.openbmc.project.Common.Error.InternalFailure Aug 27, 2022 5:24:36 AM
HIGH Error: open_power.Host.Boot.Error.Checkstop Aug 27, 2022 4:41:00 AM
HIGH Error: open_power.Host.Error.MaintenanceProcecure Aug 27, 2022 4:40:47 AM
HIGH Error: open_power.Host.Error.Event Aug 27, 2022 4:40:46 AM

I say possibly because these errors are old and have been in the log since before this boot issue manifested itself.

Troubleshoot:

1) Re-seat all components, cables
Reseated RAM, CPU, fan connectors, power connectors and tested with PCI devices and all rear+front I/O disconnected - issue persists

2) Check power supply
Checked power supply health with a tester - PSU is functioning properly
Attempted bootup with a different PSU (also tested functional) - issue persists

3) Reflash PNOR, BMC
-Pending-

This Blackbird board typically only sees occasional use for testing, with the only recent change being that I'd started leaving mains power (behind a UPS) connected 24/7 when it is between usage. It is upsetting to have boards costing north of a thousand USD just randomly quit on me, though I will try my best to recover it. Any input is welcome.