Skip to content

Commit 11cb1ce

Browse files
authored
Update the microgrid component graph to v0.5.0 and make it configurable (#1437)
The microgrid component graph library v0.5.0 changes the per-category formulas to read the components first and use the meter as the fallback. This PR upgrades the SDK to it and lets applications configure the component graph from the top-level API. ## Changes - Update `frequenz-microgrid-component-graph` to `>= 0.5.0, < 0.6`. - Add a `component_graph_config` keyword argument to `microgrid.initialize()`. It takes a `ComponentGraphConfig`, which is passed to the `ComponentGraph` constructor. `ComponentGraphConfig` and `FormulaOverrides` are re-exported from `frequenz.sdk.microgrid`. - Update the formula tests to send component values that differ from the meter values, so both the new component-first order and the meter fallback are verified at runtime. - Rename the test helper `ComponentGraphConfig` to `ComponentGraphSpec`, to avoid confusion with the re-exported library class. ## Breaking - Per-category formulas (battery, PV, CHP, EV charger, ...) now read the components first and use the meter as the fallback. Pass `ComponentGraphConfig(prefer_meters_in_component_formulas=True)` to `microgrid.initialize()` to restore the old order. - Generated formula strings can differ from the previous release, and graph validation errors use the new `Graph validation failed:` message format.
2 parents 06c5d19 + 96faca3 commit 11cb1ce

10 files changed

Lines changed: 173 additions & 37 deletions

File tree

‎RELEASE_NOTES.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,19 @@
11
# Frequenz Python SDK Release Notes
22

3+
## Summary
4+
5+
<!-- Here goes a general summary of what this release is about -->
6+
37
## Upgrading
48

5-
* Update microgrid client to v0.18.3+, which fixes a problem with missing steam boilers on formula generation.
6-
* The PV inverter manager now subtracts the power measured on unreachable PV inverters from the distribution target, so the power sent to the reachable inverters can change when some requested PV inverters are unreachable.
9+
* Update the microgrid component graph library to v0.5.0. The per-category formulas (battery, PV, CHP, EV charger, ...) now read the component first and use the meter as the fallback. This is the opposite of the old order, so generated formula strings and power values can change. To keep the old order, pass `ComponentGraphConfig(prefer_meters_in_component_formulas=True)` to `microgrid.initialize()`.
10+
11+
* Graph validation errors now use a new message format.
712

813
## New Features
914

10-
* The PV inverter manager now accounts for the power measured on unreachable PV inverters when distributing power, so the reachable inverters compensate for it (matching the battery manager's behavior).
15+
* `microgrid.initialize()` has a new keyword argument `component_graph_config`. It takes a `ComponentGraphConfig` and gives full control over how the component graph is built. For example, pass `ComponentGraphConfig(prefer_meters_in_component_formulas=True)` to make the per-category formulas read from the meter first, as in previous releases. `ComponentGraphConfig` and `FormulaOverrides` are re-exported from `frequenz.sdk.microgrid`.
1116

1217
## Bug Fixes
1318

14-
* Fixed a resource leak in the power distributor: the formulas created for unreachable batteries were never stopped, so CPU usage slowly climbed over time until the application was restarted.
15-
* Fixed the grid reactive-power formula being recreated and leaked on every access instead of being reused from the cache.
19+
<!-- Here goes notable bug fixes that are worth a special mention or explanation -->

‎pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ dependencies = [
3030
# changing the version
3131
# (plugins.mkdocstrings.handlers.python.import)
3232
"frequenz-client-microgrid >= 0.18.3, < 0.19.0",
33-
"frequenz-microgrid-component-graph >= 0.4, < 0.5",
33+
"frequenz-microgrid-component-graph >= 0.5.0, < 0.6",
3434
"frequenz-client-common >= 0.3.6, < 0.5.0",
3535
"frequenz-channels >= 1.6.1, < 2.0.0",
3636
"frequenz-quantities[marshmallow] >= 1.0.0, < 2.0.0",

‎src/frequenz/sdk/microgrid/__init__.py‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -370,6 +370,8 @@
370370

371371
from datetime import timedelta
372372

373+
from frequenz.microgrid_component_graph import ComponentGraphConfig, FormulaOverrides
374+
373375
from ..timeseries._resampling._config import ResamplerConfig
374376
from . import _data_pipeline, connection_manager
375377
from ._data_pipeline import (
@@ -393,6 +395,7 @@ async def initialize(
393395
api_power_request_timeout: timedelta = timedelta(seconds=5.0),
394396
# pylint: disable-next: line-too-long
395397
battery_power_manager_algorithm: PowerManagerAlgorithm = PowerManagerAlgorithm.SHIFTING_MATRYOSHKA, # noqa: E501
398+
component_graph_config: ComponentGraphConfig | None = None,
396399
) -> None:
397400
"""Initialize the microgrid connection manager and the data pipeline.
398401
@@ -409,8 +412,17 @@ async def initialize(
409412
will be unavailable from the corresponding component pools.
410413
battery_power_manager_algorithm: The power manager algorithm to use for
411414
batteries.
415+
component_graph_config: The configuration for building the component
416+
graph. Use it, for example, to make the per-category formulas
417+
(like the battery or PV formulas) read from the meter first
418+
(`prefer_meters_in_component_formulas`), or to set per-formula
419+
overrides through `FormulaOverrides`. When `None`, the library's
420+
default configuration is used.
412421
"""
413-
await connection_manager.initialize(server_url)
422+
await connection_manager.initialize(
423+
server_url,
424+
component_graph_config=component_graph_config,
425+
)
414426
await _data_pipeline.initialize(
415427
resampler_config,
416428
api_power_request_timeout=api_power_request_timeout,
@@ -419,6 +431,8 @@ async def initialize(
419431

420432

421433
__all__ = [
434+
"ComponentGraphConfig",
435+
"FormulaOverrides",
422436
"PowerManagerAlgorithm",
423437
"initialize",
424438
"consumer",

‎src/frequenz/sdk/microgrid/connection_manager.py‎

Lines changed: 36 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
MicrogridApiClient,
1919
MicrogridInfo,
2020
)
21+
from frequenz.microgrid_component_graph import ComponentGraphConfig
2122

2223
from .component_graph import ComponentGraph
2324

@@ -27,7 +28,12 @@
2728
class ConnectionManager(ABC):
2829
"""Creates and stores core features."""
2930

30-
def __init__(self, server_url: str) -> None:
31+
def __init__(
32+
self,
33+
server_url: str,
34+
*,
35+
component_graph_config: ComponentGraphConfig | None = None,
36+
) -> None:
3137
"""Create object instance.
3238
3339
Args:
@@ -36,9 +42,12 @@ def __init__(self, server_url: str) -> None:
3642
where the port should be an int between `0` and `65535` (defaulting to
3743
`9090`) and ssl should be a boolean (defaulting to false). For example:
3844
`grpc://localhost:1090?ssl=true`.
45+
component_graph_config: The configuration for building the component
46+
graph. When `None`, the library's default configuration is used.
3947
"""
4048
super().__init__()
4149
self._server_url = server_url
50+
self._component_graph_config = component_graph_config
4251

4352
@property
4453
def server_url(self) -> str:
@@ -94,7 +103,12 @@ async def _initialize(self) -> None:
94103
class _InsecureConnectionManager(ConnectionManager):
95104
"""Microgrid Api with insecure channel implementation."""
96105

97-
def __init__(self, server_url: str) -> None:
106+
def __init__(
107+
self,
108+
server_url: str,
109+
*,
110+
component_graph_config: ComponentGraphConfig | None = None,
111+
) -> None:
98112
"""Create and stores core features.
99113
100114
Args:
@@ -103,8 +117,10 @@ def __init__(self, server_url: str) -> None:
103117
where the port should be an int between `0` and `65535` (defaulting to
104118
`9090`) and ssl should be a boolean (defaulting to false). For example:
105119
`grpc://localhost:1090?ssl=true`.
120+
component_graph_config: The configuration for building the component
121+
graph. When `None`, the library's default configuration is used.
106122
"""
107-
super().__init__(server_url)
123+
super().__init__(server_url, component_graph_config=component_graph_config)
108124
self._client = MicrogridApiClient(server_url)
109125
# To create graph from the API client we need await.
110126
# So create empty graph here, and update it in `run` method.
@@ -172,14 +188,23 @@ async def _initialize(self) -> None:
172188
self._client.list_components(),
173189
self._client.list_connections(),
174190
)
175-
self._graph = ComponentGraph(set(components), set(connections))
191+
if self._component_graph_config is None:
192+
self._graph = ComponentGraph(set(components), set(connections))
193+
else:
194+
self._graph = ComponentGraph(
195+
set(components), set(connections), self._component_graph_config
196+
)
176197

177198

178199
_CONNECTION_MANAGER: ConnectionManager | None = None
179200
"""The ConnectionManager singleton instance."""
180201

181202

182-
async def initialize(server_url: str) -> None:
203+
async def initialize(
204+
server_url: str,
205+
*,
206+
component_graph_config: ComponentGraphConfig | None = None,
207+
) -> None:
183208
"""Initialize the MicrogridApi. This function should be called only once.
184209
185210
Args:
@@ -188,6 +213,8 @@ async def initialize(server_url: str) -> None:
188213
where the port should be an int between `0` and `65535` (defaulting to
189214
`9090`) and ssl should be a boolean (defaulting to false). For example:
190215
`grpc://localhost:1090?ssl=true`.
216+
component_graph_config: The configuration for building the component
217+
graph. When `None`, the library's default configuration is used.
191218
"""
192219
# From Doc: pylint just try to discourage this usage.
193220
# That doesn't mean you cannot use it.
@@ -197,7 +224,10 @@ async def initialize(server_url: str) -> None:
197224

198225
_logger.info("Connecting to microgrid at %s", server_url)
199226

200-
connection_manager = _InsecureConnectionManager(server_url)
227+
connection_manager = _InsecureConnectionManager(
228+
server_url,
229+
component_graph_config=component_graph_config,
230+
)
201231
await connection_manager._initialize() # pylint: disable=protected-access
202232

203233
# Check again that _MICROGRID_API is None in case somebody had the great idea of

‎tests/microgrid/test_microgrid_api.py‎

Lines changed: 77 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
Meter,
2929
)
3030

31-
from frequenz.sdk.microgrid import connection_manager
31+
from frequenz.sdk.microgrid import ComponentGraphConfig, connection_manager
3232

3333
_MICROGRID_ID = MicrogridId(1)
3434

@@ -135,6 +135,28 @@ def microgrid(self) -> MicrogridInfo:
135135
),
136136
)
137137

138+
@staticmethod
139+
def _mock_microgrid_client(
140+
components: list[list[Component]],
141+
connections: list[list[ComponentConnection]],
142+
microgrid: MicrogridInfo,
143+
) -> MagicMock:
144+
"""Create a mock microgrid API client.
145+
146+
Args:
147+
components: components to return, one list per call.
148+
connections: connections to return, one list per call.
149+
microgrid: the information about the microgrid.
150+
151+
Returns:
152+
the mock client.
153+
"""
154+
client = MagicMock()
155+
client.list_components = AsyncMock(side_effect=components)
156+
client.list_connections = AsyncMock(side_effect=connections)
157+
client.get_microgrid_info = AsyncMock(return_value=microgrid)
158+
return client
159+
138160
@mock.patch("grpc.aio.insecure_channel")
139161
async def test_connection_manager(
140162
self,
@@ -151,10 +173,9 @@ async def test_connection_manager(
151173
connections: connections
152174
microgrid: the information about the microgrid
153175
"""
154-
microgrid_client = MagicMock()
155-
microgrid_client.list_components = AsyncMock(side_effect=components)
156-
microgrid_client.list_connections = AsyncMock(side_effect=connections)
157-
microgrid_client.get_microgrid_info = AsyncMock(return_value=microgrid)
176+
microgrid_client = self._mock_microgrid_client(
177+
components, connections, microgrid
178+
)
158179

159180
with mock.patch(
160181
"frequenz.sdk.microgrid.connection_manager.MicrogridApiClient",
@@ -214,6 +235,57 @@ async def test_connection_manager(
214235
assert api.microgrid_id == microgrid.id
215236
assert api.location == microgrid.location
216237

238+
@mock.patch("grpc.aio.insecure_channel")
239+
async def test_component_graph_config(
240+
self,
241+
_insecure_channel_mock: MagicMock,
242+
components: list[list[Component]],
243+
connections: list[list[ComponentConnection]],
244+
microgrid: MicrogridInfo,
245+
) -> None:
246+
"""Test that the component graph config is used to build the graph.
247+
248+
Args:
249+
_insecure_channel_mock: insecure channel mock from `mock.patch`
250+
components: components
251+
connections: connections
252+
microgrid: the information about the microgrid
253+
"""
254+
microgrid_client = self._mock_microgrid_client(
255+
[components[0]] * 2, [connections[0]] * 2, microgrid
256+
)
257+
258+
with mock.patch(
259+
"frequenz.sdk.microgrid.connection_manager.MicrogridApiClient",
260+
return_value=microgrid_client,
261+
):
262+
# pylint: disable=protected-access
263+
default_manager = connection_manager._InsecureConnectionManager(
264+
"grpc://127.0.0.1:10001"
265+
)
266+
await default_manager._initialize()
267+
268+
meters_first_manager = connection_manager._InsecureConnectionManager(
269+
"grpc://127.0.0.1:10001",
270+
component_graph_config=ComponentGraphConfig(
271+
prefer_meters_in_component_formulas=True
272+
),
273+
)
274+
await meters_first_manager._initialize()
275+
276+
batteries = {ComponentId(9), ComponentId(12)}
277+
# By default the battery inverters are the primary source and the
278+
# meters are the fallback.
279+
assert (
280+
default_manager.component_graph.battery_formula(batteries)
281+
== "COALESCE(#8, #7, 0.0) + COALESCE(#11, #10, 0.0)"
282+
)
283+
# With `prefer_meters_in_component_formulas` the meters come first.
284+
assert (
285+
meters_first_manager.component_graph.battery_formula(batteries)
286+
== "COALESCE(#7, #8, 0.0) + COALESCE(#10, #11, 0.0)"
287+
)
288+
217289
@mock.patch("grpc.aio.insecure_channel")
218290
async def test_connection_manager_another_method(
219291
self,

‎tests/timeseries/_battery_pool/test_battery_pool.py‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@
4848
from ...utils.component_data_streamer import MockComponentDataStreamer
4949
from ...utils.component_data_wrapper import BatteryDataWrapper, InverterDataWrapper
5050
from ...utils.component_graph_utils import (
51-
ComponentGraphConfig,
51+
ComponentGraphSpec,
5252
create_component_graph_structure,
5353
)
5454
from ...utils.mock_microgrid_client import MockMicrogridClient
@@ -113,7 +113,7 @@ class SetupArgs:
113113

114114

115115
def create_mock_microgrid(
116-
mocker: MockerFixture, config: ComponentGraphConfig
116+
mocker: MockerFixture, config: ComponentGraphSpec
117117
) -> MockMicrogridClient:
118118
"""Create mock microgrid and initialize it.
119119
@@ -143,9 +143,7 @@ async def setup_all_batteries(mocker: MockerFixture) -> AsyncIterator[SetupArgs]
143143
Yields:
144144
Arguments that are needed in test.
145145
"""
146-
mock_microgrid = create_mock_microgrid(
147-
mocker, ComponentGraphConfig(batteries_num=2)
148-
)
146+
mock_microgrid = create_mock_microgrid(mocker, ComponentGraphSpec(batteries_num=2))
149147
min_update_interval: float = 0.2
150148
# pylint: disable=protected-access
151149
microgrid._data_pipeline._DATA_PIPELINE = None
@@ -195,9 +193,7 @@ async def setup_batteries_pool(mocker: MockerFixture) -> AsyncIterator[SetupArgs
195193
Yields:
196194
Arguments that are needed in test.
197195
"""
198-
mock_microgrid = create_mock_microgrid(
199-
mocker, ComponentGraphConfig(batteries_num=4)
200-
)
196+
mock_microgrid = create_mock_microgrid(mocker, ComponentGraphSpec(batteries_num=4))
201197
streamer = MockComponentDataStreamer(mock_microgrid)
202198
min_update_interval: float = 0.2
203199
# pylint: disable=protected-access
@@ -520,7 +516,21 @@ async def test_battery_pool_power(mocker: MockerFixture) -> None:
520516
power_receiver = battery_pool.power.new_receiver()
521517

522518
# send meter power [grid_meter, battery1_meter, battery2_meter]
519+
# and battery inverter power [battery1_inverter, battery2_inverter].
520+
# The inverter values differ from the meter values, to check that
521+
# the inverters are the primary source.
522+
await mockgrid.mock_resampler.send_meter_power([100.0, 20.0, 30.0])
523+
await mockgrid.mock_resampler.send_bat_inverter_power([21.0, 31.0])
524+
assert (await power_receiver.receive()).value == Power.from_watts(52.0)
525+
526+
# When the inverters have no value, the formula falls back to the
527+
# meters. The fallback subscription starts after the first `None`,
528+
# so send twice and skip one output.
529+
await mockgrid.mock_resampler.send_meter_power([100.0, 20.0, 30.0])
530+
await mockgrid.mock_resampler.send_bat_inverter_power([None, None])
523531
await mockgrid.mock_resampler.send_meter_power([100.0, 20.0, 30.0])
532+
await mockgrid.mock_resampler.send_bat_inverter_power([None, None])
533+
await power_receiver.receive()
524534
assert (await power_receiver.receive()).value == Power.from_watts(50.0)
525535

526536

‎tests/timeseries/_ev_charger_pool/test_ev_charger_pool.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,11 @@ async def test_ev_power( # pylint: disable=too-many-locals
4646
ev_pool = microgrid.new_ev_charger_pool(priority=5)
4747
power_receiver = ev_pool.power.new_receiver()
4848

49+
# The charger values sum to 15 W, not the 16 W on the meter, to
50+
# check that the chargers are the primary source.
4951
await mockgrid.mock_resampler.send_meter_power([16.0])
50-
assert (await power_receiver.receive()).value == Power.from_watts(16.0)
52+
await mockgrid.mock_resampler.send_evc_power([2.0, 6.0, 7.0])
53+
assert (await power_receiver.receive()).value == Power.from_watts(15.0)
5154

5255

5356
async def test_power_status_same_instance_subscriptions_work(

‎tests/timeseries/_formulas/test_composition.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -232,13 +232,13 @@ async def test_formula_composition_min_max(self, mocker: MockerFixture) -> None:
232232
assert (
233233
str(formula_min)
234234
== "[grid_power_min]("
235-
+ "MIN([grid_power](#4), [chp_power](COALESCE(#7, #5, 0.0)))"
235+
+ "MIN([grid_power](#4), [chp_power](COALESCE(#5, #7, 0.0)))"
236236
+ ")"
237237
)
238238
assert (
239239
str(formula_max)
240240
== "[grid_power_max]("
241-
+ "MAX([grid_power](#4), [chp_power](COALESCE(#7, #5, 0.0)))"
241+
+ "MAX([grid_power](#4), [chp_power](COALESCE(#5, #7, 0.0)))"
242242
+ ")"
243243
)
244244

‎tests/timeseries/test_logical_meter.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,5 +49,8 @@ async def test_pv_power(self, mocker: MockerFixture) -> None:
4949
stack.push_async_callback(pv_pool.stop)
5050
pv_power_receiver = pv_pool.power.new_receiver()
5151

52+
# The inverter values differ from the meter values, to check
53+
# that the inverters are the primary source.
5254
await mockgrid.mock_resampler.send_meter_power([-10.0, -20.0])
53-
assert (await pv_power_receiver.receive()).value == Power.from_watts(-30.0)
55+
await mockgrid.mock_resampler.send_pv_inverter_power([-9.0, -19.0])
56+
assert (await pv_power_receiver.receive()).value == Power.from_watts(-28.0)

0 commit comments

Comments
 (0)