Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium (github.com)

19 points by sanchitmonga22 3 hours ago

2 comments:

by bookernath an hour ago

Tokens per minute is a strange metric, I usually see tok/s. A per-second figure would be more accessible in comparing this to alternatives.

by badmonster0 35 minutes ago

how does this compare with sglang

Data from: Hacker News, provided by Hacker News (unofficial) API