331
Software is slow because of bad design, not slow languages
(www.seangoedecke.com)
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Follow the wormhole through a path of communities !webdev@programming.dev
Profiling complex systems can be quite difficult. That said, I believe in the adage "Profile twice, optimize once.", adapted from the old carpenters adage of "Measure twice, cut once."
Slowness is rarely essential, but I can be far away from where you expect it.
That's what Open Telemetry is for, honestly, specifically tracing. I'm in the process of adding tracing for all of our web handlers, http client calls, database transactions/individual queries, and asynchronous jobs. Should give us a pretty good view into the performance of the system, and allow us to monitor for things like performance regressions without any dedicated profiling needed (though we certainly could add additional profiling information through Otel down the road). Since I know you're a Haskeller (recognize your handle), the
hs-opentelemetryecosystem is quite solid and has integrations with a lot of libraries in the wider ecosystem, likewai,persistent,postgresql-simple,conduit, and a bunch more. I haven't needed to write too much of our own instrumentation, aside from our job system since that's a bespoke system.Yeah we used hs-opentelemetry at my last paid position. The company was an MS shop, tho so we were using ODBC to connect to MS SQL Server which doesn't have great in-process profiling. But, at least we could figure out what quer(y/ies) were running long and then we could use the MS tools to look at the plan and fiddle with that. It was also a small org, so it's not like there was a lot of internal services in one flow -- basically everything either stayed in the Web Yesod app or in the IVR app. All our profiling could have been instrumented process profiling instead of having to use otel at all.
So, I haven't seen how easy/difficult it is to actually use otel across multiple internal service calls to satisfy one external request.