Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration system implementation #78

Merged
merged 23 commits into from
Sep 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions docs/tutorial/impacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,12 +155,19 @@ We use impact factors to quantify environmental harm from human activities, meas

### Electricity Mix

**We currently assume by default a worldwide average impact factor for electricity consumption**. We plan to allow users to change these impact factors dynamically based on a specific country/region or with custom values.
When initializing `EcoLogits`, you can choose a specific electricity mix zone from the [ADEME Base Empreinte® :octicons-link-external-16:](https://base-empreinte.ademe.fr/) database.

Default values (from [ADEME Base Empreinte®](https://base-empreinte.ademe.fr/)):
```python title="Select a different electricity mix"
from ecologits import EcoLogits

# Select the electricity mix of France
EcoLogits.init(electricity_mix_zone="FRA")
```

By default, the `WOR` World electricity mix is used, whose values are:

| Impact criteria | Value | Unit |
|-----------------|------------|-----------------|
| GWP | $5.904e-1$ | $kgCO2eq / kWh$ |
| ADPe | $7.378e-7$ | $kgSbeq / kWh$ |
| PE | $9.988$ | $MJ / kWh$ |
| GWP | $5.904e-1$ | kgCO2eq / kWh |
| ADPe | $7.378e-7$ | kgSbeq / kWh |
| PE | $9.988$ | MJ / kWh |
45 changes: 36 additions & 9 deletions docs/tutorial/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,21 +45,48 @@ It achieves this by **patching the Python client libraries**, ensuring that each

## Initialization of EcoLogits

To use EcoLogits in your projects, you will need to initialize the client tracers that are used internally to intercept and enrich responses.

!!! info "Default behavior is to search and initialize all available providers."
To use EcoLogits in your projects, you will need to initialize the client tracers that are used internally to intercept and enrich responses. The default initialization will use default parameters and enable tracking of all available providers. To change that behaviour, read along on how to configure EcoLogits.

```python
from ecologits import EcoLogits

# Initialize for all available providers
# Default initialization method
EcoLogits.init()
```


### Configure providers

You can select which provider to enable with EcoLogits using the `providers` parameter.

# Initialize for `openai` provider only
EcoLogits.init("openai")
!!! info "Default behavior is to enable all available providers."

# Initialize for `openai` and `anthropic` providers only
EcoLogits.init(["openai", "anthropic"])
```python title="Select a providers to enable"
from ecologits import EcoLogits

# Examples on how to enable one or multiple providers
EcoLogits.init(providers="openai")
EcoLogits.init(providers=["anthropic", "mistralai"])
```

It is currently not possible to un-initialize a provider at runtime. If that's the case do not hesitate to [open an issue :octicons-link-external-16:](https://github.com/genai-impact/ecologits/issues/new/choose) and explain why it could be necessary for your use case.
??? warning "Disabling a provider at runtime is not supported"

**It is currently not possible to dynamically activate and deactivate a provider at runtime.** Each time that `EcoLogits` is re-initialized with another providers, the latter will be added to the list of already initialized providers. If you think that un-initializing a provider could be necessary for your use case, please [open an issue :octicons-link-external-16:](https://github.com/genai-impact/ecologits/issues/new/choose)."


### Configure electricity mix

You can change the [electricity mix :octicons-link-external-16:](https://ourworldindata.org/electricity-mix) for server-side computation depending on a specific location. EcoLogits will automatically change the default impact factors for electricity consumption according to the selected zone.

Available zones are listed in the [electricity_mixes.csv :octicons-link-external-16:](https://github.com/genai-impact/ecologits/blob/main/ecologits/data/electricity_mixes.csv) file and are based on the [ISO 3166-1 alpha-3 :octicons-link-external-16:](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-3) convention with some extras like `WOR` for World or `EEE` for Europe.

Electricity mixes for each geographic zone are sourced from the [ADEME Base Empreinte® :octicons-link-external-16:](https://base-empreinte.ademe.fr/) database and are based on yearly averages.

!!! info "Default electricity mix zone is `WOR` for World."

```python title="Select a different electricity mix"
from ecologits import EcoLogits

# Select the electricity mix of France
EcoLogits.init(electricity_mix_zone="FRA")
```
36 changes: 24 additions & 12 deletions ecologits/_ecologits.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import importlib.util
from typing import Union
from dataclasses import dataclass, field
from typing import Optional, Union

from packaging.version import Version

Expand Down Expand Up @@ -106,30 +107,41 @@ class EcoLogits:
```

"""
@dataclass
class _Config:
electricity_mix_zone: str = field(default="WOR")
providers: list[str] = field(default_factory=list)

initialized = False
config = _Config()

@staticmethod
def init(providers: Union[str, list[str]] = None) -> None:
def init(
providers: Optional[Union[str, list[str]]] = None,
electricity_mix_zone: Optional[str] = "WOR",
) -> None:
"""
Initialization static method. Will attempt to initialize all providers by default.

Args:
providers: list of providers to initialize.
providers: list of providers to initialize (all providers by default).
electricity_mix_zone: ISO 3166-1 alpha-3 code of the electricity mix zone (WOR by default).
"""
if providers is None:
providers = list(_INSTRUMENTS.keys())
if isinstance(providers, str):
providers = [providers]
if not EcoLogits.initialized:
init_instruments(providers)
EcoLogits.initialized = True
if providers is None:
providers = list(_INSTRUMENTS.keys())

init_instruments(providers)

EcoLogits.config.electricity_mix_zone = electricity_mix_zone
EcoLogits.config.providers += providers
EcoLogits.config.providers = list(set(EcoLogits.config.providers))


def init_instruments(providers: list[str]) -> None:
for provider in providers:
if provider not in _INSTRUMENTS:
raise EcoLogitsError(f"Could not find tracer for the `{provider}` provider.")

init_func = _INSTRUMENTS[provider]
init_func()
if provider not in EcoLogits.config.providers:
init_func = _INSTRUMENTS[provider]
init_func()
1 change: 1 addition & 0 deletions ecologits/electricity_mix_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,5 @@ def from_csv(cls, filepath: Optional[str] = None) -> "ElectricityMixRepository":
)
return cls(electricity_mixes)


electricity_mixes = ElectricityMixRepository.from_csv()
3 changes: 3 additions & 0 deletions ecologits/log.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import logging

logger = logging.getLogger(__name__)
5 changes: 5 additions & 0 deletions ecologits/tracers/anthropic_tracer.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from typing_extensions import override
from wrapt import wrap_function_wrapper

from ecologits._ecologits import EcoLogits
from ecologits.impacts import Impacts
from ecologits.tracers.utils import llm_impacts

Expand Down Expand Up @@ -59,6 +60,7 @@ def __stream_text__(self) -> Iterator[str]:
model_name=model_name,
output_token_count=output_tokens,
request_latency=requests_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)

def __init__(self, parent) -> None: # noqa: ANN001
Expand Down Expand Up @@ -92,6 +94,7 @@ async def __stream_text__(self) -> AsyncIterator[str]:
model_name=model_name,
output_token_count=output_tokens,
request_latency=requests_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)

def __init__(self, parent) -> None: # noqa: ANN001
Expand Down Expand Up @@ -154,6 +157,7 @@ def anthropic_chat_wrapper(
model_name=model_name,
output_token_count=response.usage.output_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
return Message(**response.model_dump(), impacts=impacts)
Expand All @@ -173,6 +177,7 @@ async def anthropic_async_chat_wrapper(
model_name=model_name,
output_token_count=response.usage.output_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
return Message(**response.model_dump(), impacts=impacts)
Expand Down
5 changes: 5 additions & 0 deletions ecologits/tracers/cohere_tracer.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

from wrapt import wrap_function_wrapper

from ecologits._ecologits import EcoLogits
from ecologits.impacts import Impacts
from ecologits.tracers.utils import llm_impacts

Expand Down Expand Up @@ -50,6 +51,7 @@ def cohere_chat_wrapper(
model_name=model_name,
output_token_count=output_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
return NonStreamedChatResponse(**response.dict(), impacts=impacts)

Expand All @@ -67,6 +69,7 @@ async def cohere_async_chat_wrapper(
model_name=model_name,
output_token_count=output_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
return NonStreamedChatResponse(**response.dict(), impacts=impacts)

Expand All @@ -86,6 +89,7 @@ def cohere_stream_chat_wrapper(
model_name=model_name,
output_token_count=output_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
yield StreamedChatResponse_StreamEnd(**event.dict(), impacts=impacts)
else:
Expand All @@ -107,6 +111,7 @@ async def cohere_async_stream_chat_wrapper(
model_name=model_name,
output_token_count=output_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
yield StreamedChatResponse_StreamEnd(**event.dict(), impacts=impacts)
else:
Expand Down
5 changes: 5 additions & 0 deletions ecologits/tracers/google_tracer.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

from wrapt import wrap_function_wrapper

from ecologits._ecologits import EcoLogits
from ecologits.tracers.utils import llm_impacts

try:
Expand Down Expand Up @@ -91,6 +92,7 @@ def google_chat_wrapper_non_stream(
model_name=model_name, # ?
output_token_count=response.usage_metadata.total_token_count,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone,
)
if impacts is not None:
# Convert the response object to a dictionary (model_dump() is not available in the response object)
Expand All @@ -114,6 +116,7 @@ def google_chat_wrapper_stream(
model_name=model_name, # ?
output_token_count=chunk.usage_metadata.total_token_count,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
chunk = wrap_from_dict(chunk.__dict__, impacts) # noqa: PLW2901
Expand Down Expand Up @@ -144,6 +147,7 @@ async def google_async_chat_wrapper_non_stream(
model_name=model_name, # ?
output_token_count=response.usage_metadata.total_token_count,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
# Convert the response object to a dictionary (model_dump() is not available in the response object)
Expand All @@ -167,6 +171,7 @@ async def google_async_chat_wrapper_stream(
model_name=model_name, # ?
output_token_count=chunk.usage_metadata.total_token_count,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
chunk = wrap_from_dict(chunk.__dict__, impacts, async_mode = True) # noqa: PLW2901
Expand Down
13 changes: 9 additions & 4 deletions ecologits/tracers/huggingface_tracer.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

from wrapt import wrap_function_wrapper

from ecologits._ecologits import EcoLogits
from ecologits.impacts import Impacts
from ecologits.tracers.utils import llm_impacts

Expand Down Expand Up @@ -66,7 +67,8 @@ def huggingface_chat_wrapper_non_stream(
provider=PROVIDER,
model_name=instance.model,
output_token_count=output_tokens,
request_latency=request_latency
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
return ChatCompletionOutput(**asdict(response), impacts=impacts)
Expand All @@ -90,7 +92,8 @@ def huggingface_chat_wrapper_stream(
provider=PROVIDER,
model_name=instance.model,
output_token_count=token_count,
request_latency=request_latency
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
yield ChatCompletionStreamOutput(**asdict(chunk), impacts=impacts)
Expand Down Expand Up @@ -125,7 +128,8 @@ async def huggingface_async_chat_wrapper_non_stream(
provider=PROVIDER,
model_name=instance.model,
output_token_count=output_tokens,
request_latency=request_latency
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
return ChatCompletionOutput(**asdict(response), impacts=impacts)
Expand All @@ -149,7 +153,8 @@ async def huggingface_async_chat_wrapper_stream(
provider=PROVIDER,
model_name=instance.model,
output_token_count=token_count,
request_latency=request_latency
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
yield ChatCompletionStreamOutput(**asdict(chunk), impacts=impacts)
Expand Down
5 changes: 5 additions & 0 deletions ecologits/tracers/litellm_tracer.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

from wrapt import wrap_function_wrapper

from ecologits._ecologits import EcoLogits
from ecologits.impacts import Impacts
from ecologits.model_repository import models
from ecologits.tracers.utils import llm_impacts
Expand Down Expand Up @@ -59,6 +60,7 @@ def litellm_chat_wrapper_stream(
model_name=model_name,
output_token_count=token_count,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone,
)
if impacts is not None:
yield ChatCompletionChunk(**chunk.model_dump(), impacts=impacts)
Expand All @@ -81,6 +83,7 @@ def litellm_chat_wrapper_non_stream(
model_name=model_name,
output_token_count=response.usage.completion_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
return ChatCompletion(**response.model_dump(), impacts=impacts)
Expand Down Expand Up @@ -115,6 +118,7 @@ async def litellm_async_chat_wrapper_base(
model_name=model_name,
output_token_count=response.usage.completion_tokens,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
return ChatCompletion(**response.model_dump(), impacts=impacts)
Expand Down Expand Up @@ -142,6 +146,7 @@ async def litellm_async_chat_wrapper_stream(
model_name=model_name,
output_token_count=token_count,
request_latency=request_latency,
electricity_mix_zone=EcoLogits.config.electricity_mix_zone
)
if impacts is not None:
yield ChatCompletionChunk(**chunk.model_dump(), impacts=impacts)
Expand Down
Loading
Loading