Widespread and extended supply blackouts struck the island of Newfoundland in January 2013, January 2014, and March 2015. Each time, in the order of half the island was blacked out with many customers in the freezing darkness for more than 48 hours. A report from the Newfoundland & Labrador Board of Commissioners of Public Utilities — often called the PUB — issued September 29 found widespread deficiencies within the provincial government-owned energy holding company Nalcor’s electricity operation, Newfoundland Hydro.

Outside of Newfoundland, there may be few people who care much about these events. I suggest however, that anyone who cares about the standards of good utility practice and also good utility regulator practice spend the few minutes it will take to read at least a couple of paragraphs of the next 1700ish words in this long-winded post.

Electricity service on the island of Newfoundland is provided by the government-owned generation and transmission utility Newfoundland Hydro and by the privately owned distribution utility Newfoundland Power.

The PUB’s report determined that the outages originated on Newfoundland Hydro’s system (not Newfoundland Power) and were the result of “Hydro’s failure to effectively plan and manage its assets and that Hydro failed to meet the standard of generally accepted sound public utility practice.” Another conclusion of the report was that “It is concerning that Hydro was found to be imprudent with respect to so many of the identified projects and activities and further that the imprudence was found to have occurred over the course of a number of years.” The Board also concluded that “there are significant continuing risks to the adequacy and reliability of supply on the Island Interconnected system.”

Barring bad nuclear problems in New Brunswick or Ontario, Newfoundland and Labrador has by far the worst electricity outlook in Canada. Ghastly unreliability isn’t the only problem. Official Newfoundland and Labrador has acknowledged that rates will approximately double when the province’s first big project sold to the public as green energy, Muskrat Falls, comes into service at the end of this decade or early in the next decade. This, notwithstanding a unique financial structure for repayment of much of the Muskrat Falls project that back-end loads recovery of Muskrat costs up to 50 years into the future.

Two days after power was completely restored on January 8, 2014, the PUB started a public inquiry on its own initiative into the blackouts. Inconsistent with practices becoming more common in Canadian utility circles, the PUB retained a competent and independent expert advisor, the Liberty Consulting Group based in Lebanon, PA. Liberty provided two major reports as well as expert testimony.

The inquiry received public input in written submissions and oral hearings. Much of the citizen input was from members of citizens coalition calling itself Group 2041, which has been the main force resisting the government’s Muskrat Falls project since it was launched by Conservative Premier Danny Williams just before he retired from politics in December 2010. (Other key resistance players include the Grand River Keeper and Ed Hollet who blogs here.)

The PUB’s Interim Report was issued May 15, 2014. The report concluded that there was what the PUB called an “unacceptably high risk” of outages on the island and identified immediate actions needed to reduce the risk of further outages.

The blackouts of March 2015 appeared to deepen the PUB’s resolve.

Dark Muskrat

The PUB’s report makes clear the central role that the Muskrat Falls project played in directing Newfoundland Hydro’s attention away from reliable service for islanders.

One example of how the Muskrat project diverted Newfoundland Hydro’s attention relates to maintenance for transmission protection and control systems. In 2012, Newfoundland Hydro analyzed its transmission capabilities and decided to replace obsolete relays for the transmission line between the Holyrood and Hardwoods generators during the 2013-2015 time period. Then in 2013, Hydro reversed this decision on the grounds that the future installation of transmission systems associated with Muskrat Falls would have required further changes to the Holyrood by Hardwoods line. Immediately after that decision, the line in question became a key vulnerability.

In response to persistent questioning during the inquiry about the role of the Muskrat project in contributing to blackouts, Newfoundland Hydro produced a classic bait and switch. Newfoundland Hydro submitted to the PUB that the central question is “What is the lowest reasonable level of reserve for (service to islanders) between now and 2017/18 or 2018/19 knowing that the Island will be interconnected through both Labrador and Nova Scotia once Muskrat Falls interconnections are in-service?”

In all my study of the case for the Muskrat project, nowhere have I found any acknowledgment that proceeding with the project entailed a lower level of reliability for islanders until the project was completed.

The PUB expressed particular concern about the reliability of Newfoundland Hydro’s aging thermal units in light of the announced delay in the Muskrat Falls project and evidence amassed during the inquiry demonstrating many deficiencies in Hydro’s thermal generation program.

“While it was Hydro’s view in 2015 that the Island Interconnected system can supply all customer demand under multiple contingencies through to 2018, the Board notes that a delay in the Muskrat Falls project and deteriorated generation unit performance are two of the factors which Hydro had stated may impact its assessment.The Board believes that further urgent work is required to fully assess the circumstances and risks with a view to determining the measures that are required in relation to supply. It is imperative that circumstances continue to be closely monitored so that immediate and decisive action can be taken where necessary to ensure adequate and reliable supply before interconnection with Muskrat Falls. The Board will continue to require daily system reports from Hydro in addition to its quarterly generation unit performance reports. The Board will also require continued winter readiness reporting to monitor planning, scheduling and execution. Given the serious ongoing issues, generation planning and supply will continue to be evaluated as part of Phase Two of this investigation. In the meantime, Hydro will be directed to immediately commence the supply review recommended by Liberty. (pp. 51-52)”

The PUB notes in its conclusions that “Liberty concluded that there is a continuing and high risk of supply-related emergencies until Muskrat Falls and the Labrador-Island Link come into service.”

Falling Short of Good Utility Practice

The PUB report documents many instances of carelessness and mismanagement by Newfoundland Hydro of core assets and processes critical to grid reliability.

One example relates a 50 MW diesel-fueled combustion turbine (CT) located at Stephenville. Coming into the winter of 2013-14, Newfoundland Hydro knew a potential generation shortfall was looming, in part due to problems with the Stephenville CT. Here is one of the observations noted in the PUB report about it:

In the summer of 2013, when (the Stephenville) unit was returned to service following a 20 month forced outage, it was de-rated because of worn insulating blankets. These were not replaced during the scheduled outage and Hydro did not solicit bids for the insulating blankets until October 2013.

Every fact in these two sentences raises concerns. For example, consider the length of the Stephenville forced outage. After the blackouts of January 2014, Newfoundland Hydro installed a new 120 MW CT on a green field site. The initial application was presented to the PUB in April 2014, in less than a month it was approved, and the unit was brought into service in December of that year. Another concern is that the Stephenville unit was derated due to equipment deficiencies after returning to service from a forced outage.

Newfoundland Hydro’s emergency management practices were also careless.

On the morning of January 2, 2014 Hydro’s short-term forecast indicated that the evening peak load would surpass available supply. At 2:00 p.m. Hydro issued a public advisory to request that customers on the Island Interconnected system take steps to conserve electricity where possible. Shortly after 4:00 p.m. on the same day Hydro requested that Newfoundland Power implement rotating power outages.

When a generation deficiency emergency is imminent, immediate customer advisories are in order.

Another of the PUB’s particularly informative findings was that Newfoundland Hydro had been operating for “decades” with an unreliable short-term forecasting and operating model. Liberty’s report found:

Hydro has used its current approach for decades, but its modeling, as currently constructed and used, does not produce acceptable levels of reserves.

Decades?

Newfoundland Hydro’s approach to managing its transmission assets was also found to be substandard. The PUB supported Liberty’s observation that the management approach “did not reflect appropriately the age and condition of Hydro’s assets.” Liberty documented that the resulting performance results were below that of Canadian comparators. Liberty noted that transmission reliability performance declined from 2009 to 2013 even after adjusting for the consequences of major outage events.

Newfoundland Hydro’s response to Liberty’s analysis showing a deteriorating trend was unencouraging. “While delivery point forced outage frequencies and durations increased from 2009 to 2013, there are significant variances from one year to the next.” One might ask, how could it be otherwise?

With many utilities in Canada dealing with aging transmission assets, best practices for squeezing value from these old systems should be a major focus of attention.

Other specific deficiencies identified by the PUB include: a lack of a focused worst-feeder program, persistent failure to keep up with the planned 6-year maintenance schedule for transformers, failure to service air-blast circuit breakers on schedule and also not in response to observed conditions of the equipment, field crews working without access to mobile computers, and continued reliance on obsolete manual paper-based outage management processes. If only one or two of these deficiencies existed, it might be understandable, but the sweeping scope of the deficiencies clearly indicate that there is something deeper wrong at Newfoundland Hydro.

Philosophy

A routine mechanical problem with a generator in March 2015 combined with a failure of an emergency generator to start cascaded into a widespread collapse of the grid. In trying to get to the bottom of it, Liberty found that “undisciplined, unfocused operations” created the conditions for normal equipment complications to cascade into a major collapse. For example, system planning and system operations functions within Newfoundland Hydro failed to communicate the vulnerability of the system to responsible operators and managers remedying the problems that eventually initiated the collapse. “Liberty found that, while there are many lessons to be learned from the March 2015 outage, the overriding cause was the current operating culture at Hydro, which continues to adversely influence Hydro’s decision making and contributes to operational incidents.”

Liberty identified “a culture more tolerant of rotating outages” at Hydro. In its submissions, Newfoundland Power acknowledged that, “Hydro should take specific steps to improve its operating culture vis a vis its reliability focus.”

In its report, the PUB ordered Newfoundland Hydro develop “a more robust operational philosophy
regarding reliability.” I find it hard to imagine a more damning finding coming from a utility regulator in Canada.

Concluding Note and Acknowledgement

Without the PUB’s report, it would be impossible for an outsider to figure out the depth of the problems at Newfoundland Hydro. While the thoroughness of the PUB’s work is impressive, the ability of the PUB to initiate the inquiry within two days of the restoration of service in January 2014 and the PUB’s quick and ultimately prophetic interim report are especially impressive. The report also demonstrates the importance of tribunals in having access to independent utility expertise.

Notice that Nalcor’s CEO, Ed Martin, was named Energy Person of the Year by the Energy Council of Canada in late 2013. I bet more than a few qualified professionals with a sense of public service inside Newfoundland Hydro shook their heads about that.

Without Des Sullivan’s posts on the Uncle Gnarley blog I would have missed the PUB’s report. Thank you, Des.