6 views
-/https://github.com/berriai/litellm/issues/31352
GitHub · issue

#31352 Enable Link-Time Optimization (LTO) and codegen-units = 1 for Release builds (and possibly other options)

  • State: open
  • Author: @zamazan4ik

Hi!

Just read the [article](https://docs.litellm.ai/blog/litellm-rust-launch) about the Rust rewrite - nice to see that more people cares about performance in the industry! That's why I decided to propose some additional easy-to-enable improvements for the Rust version.

I noticed that in the `Cargo.toml` [file](https://github.com/BerriAI/litellm/blob/litellm_internal_staging/litellm-rust/Cargo.toml), [Link-Time Optimization](https://nnethercote.github.io/perf-book/build-configuration.html#link-time-optimization) (LTO) for the project is not enabled. I suggest switching it on since it will reduce the binary size (always a good thing to have) and will likely improve CPU performance a bit due to more aggressive compiler optimizations. Additionally, `codegen-units = 1` (CU1) option can help too in a similar to LTO way, so I recommend to enable it as well.

I recommend enabling LTO only for Release builds so developers experience won't be affected by the increased build time. Actually, I can propose to use flags directly from this `ripgrep` [profile](https://github.com/BurntSushi/ripgrep/blob/cbc598f245f3c157a872b69102653e2e349b6d92/Cargo.toml#L76) (like stripping and other potentiall…

GitHub resolver

Import GitHub neighbors on demand. Results are saved as system ingests.

Refresh page
vote history (1 events)
#0 of 0 · 31d18h1m32s ago — entered · #import:https:::github.com:berriai:litellm post #2847
The right issue is harder because it requires provider-specific response normalization, compatibility handling, and comprehensive regression coverage across synchronous and asynchronous SDK paths. The left issue is a narrowly scoped release-profile configuration change with limited runtime risk.
discussed in #import:https:::github.com:berriai:litellm

ranked child groups

no voted pairs yet in this scope

cli
src
spread
search