Overview of Types of Relevant Knowledge
As a reminder, this section focuses on the activities and knowledge of the wholesale market operator – not those of participants. In a traditional vertically integrated utility environment, these might be the same entity, but in a deregulated environment, they are separate.
Therefore, the specialized knowledge that a market maker and transmission operator develops is not commercial knowledge about the costs and capabilities of the resources bidding into its market, but in the actual activity of running the market itself.
Several bodies of knowledge are necessary to run a wholesale market and dispatch a transmission system:
Transmission engineering knowledge
The core technical aspect of operating the transmission system is a complex, ongoing engineering problem. System operators draw on a rich and evolving engineering literature that address the issues in designing and operating transmission systems. They also obtain relevant knowledge from the manufacturers and owners of the equipment they dispatch
Market rules
The actual rules under which a wholesale market operate are a kind of knowledge. Developed through careful analysis by economists, comments from market participants, and sometimes, through learning by mistakes, market rules represent knowledge on how to run an auction that generates acceptable results. These rules are then embodied in a tariff, and further elaborated in business practice manuals. For example, during the California Energy Crisis, we learned that the existing market maker, the California PX had rules that enabled exploitation. [Here’s an explanation of how the game worked. I don’t think it’s the best ref to cite, though, since it comes from course materials: https://www.e-education.psu.edu/ebf483/node/821] [Here’s the original FERC staff report on the whole episode: https://www.ferc.gov/legal/maj-ord-reg/land-docs/PART-I-3-26-03.pdf] The CAISO market (as well as those of the other RTOs and ISOs) in part because of the CA PX’s failure.
Though there was an attempt to create a unified set of market rules for all ISOs/RTOs, it was not adopted and each ISO/RTO in the US has its own tariff. Nevertheless, ISOs/RTOs do look at how others are operating and will adopts and adapt good ideas from each other.
“Local” and territory-specific knowledge
In order to do its job effectively, an ISO/RTO must develop detailed knowledge of its own territory, including the layout and extent of the as-built transmission system, as well as its detailed electrical capabilities and characteristics. Similarly, market participants must provide the ISO/RTO with detailed knowledge of their connected resources, including their capabilities, but also schedules for planned outages, and any other limitations.
Transmission and market operators also have to develop forecast models of total demand. In theory, a transmission operator could take the demand bids from market participants as its sole source of prediction about energy needs, but in practice, transmission operators generate their own independent forecasts using complex models incorporate historical data, weather, and other factors. They do this so that they can be prepared to handle all load, even if participants under bid for some reason.
The sum total of this is all the details specific to their own territory that transmission operators / market makers must have. Even an expert coming from one ISO/RTO to another would need to learn these details to be effective.
Computer optimization
The software that runs the market optimizations along with the configuration of constraints fed into the optimization comprise a significant knowledge. The ISO/RTOs maintain the configuration and constraint knowledge necessary to run the software, the software itself usually comes from third party vendors, who have incorporated their knowledge of optimization of electric systems into it. This knowledge is therefore proprietary. Siemens in a common provider of such software to ISOs/RTOs. Further embedded in the software, core algorithms such a mixed-integer programming (MIP) are used to actually run the optimization. The knowledge of how these algorithms work is well known and there are open-source implementations, but specific high-performance implementations are usually proprietary, such as CPLEX from IBM.
Historical market results
As an ISO/RTO operates, it gains experience. Some of this experience is in the minds of its staff, but there is also a large body of numerical data that is the natural output of such an organization: the prices and dispatches generated every hour, day, etc. This price data is typically released to the public through an Internet-accessible website as soon as it is created. Dispatch information is released publicly later and in anonymized form, in order to provide temporary confidentiality to market participants.
Knowledge Spillover and Diffusion
Wholesale market knowledge diffuses and spills over in different ways and to varying degrees.
Technical knowledge in transmission engineering is quite mature and can be obtained through postsecondary education as well as books and journals. There is a vibrant and ongoing research establishment in electric transmission.
Similarly, economic expertise necessary to build a working power market is a subspecialty of microeconomics, and optimization expertise is a subspecialty of operations research. Education in such fields is readily available.
Many of the areas of knowledge of an ISO/RTO are embodied in its tariff and business practice manuals. This knowledge is available to anyone who chooses to read these public documents. Similarly, because adjustments to ISO/RTO tariffs are made through open stakeholder processes, the history and reasons for a given feature of a market design can often be found in the comments and proceedings that generated that feature.
Operational knowledge specific to a specific transmission system, of course, by definition can be obtained only through study and interaction with that system. As a result, such knowledge most likely diffuses between long-tenured and new employees within an organization.
Barriers to Knowledge Transfer
As mentioned earlier, some of the core optimization used in a wholesale market is embodied in proprietary software licensed by the ISO/RTO. As such, that knowledge will never transfer unless the owners decide to transfer it, or it is recreated independently.
Other knowledge, specific to an ISO/RTO may not transfer because there is no demand for it, and perhaps because it is not easy to encapsulate.
However, most of the knowledge used by an ISO/RTO that represents its tariff and operating principles and rules is easily transferred, because it exists in formalized text publicly available.
Market outcome knowledge is interesting because, though the data is readily available from same-time information systems, turning that data into useful knowledge often requires sophistication, effort, and some knowledge of the system from which it came. People and organization that have made the effort to build models with such data and do other “sense making” exercises, therefore have an incentive to keep their results confidential.