Ruby with jemalloc

[Vad1mo]

I would like to create a ruby version compiled with jemalloc. Is this something the community needs? Is it encouraged to make an official contribution. If, so I can try to create an official contribution.

[jonmoter]

We'd like to do the same thing, and have started down the path of building our own ruby docker image with jemalloc support. That involves a whole bunch of copy & pasting of Dockerfiles from this repo.

So it would be great if this option could be included as an option for the images published on docker hub. And I think a lot of other ruby users would find this valuable.

[Vad1mo]

@jonmoter could you open PR, in [WIP] status I will also contribute to this PR and add my stuff

[libo]

@Vad1mo @jonmoter something like this? #190

[jorihardman]

If anyone is looking to run this now, I've forked this repository, merged #192, and added a public/automated docker hub that can be found here: https://hub.docker.com/r/swipesense/ruby-jemalloc

We've been using this in production for a couple weeks now and have seen great results. I will maintain that docker hub and repo until this repo implements this request.

[shleeable]

@jorihardman amazing work.. can you open the issue register + any plans to support Alpine?

[jorihardman]

@ashleyhull-versent thanks for the alpine PR. Images are being built as a type this.

[yosifkit]

#198 (comment)

We're still hesitant to add this to the image without a recommendation by Ruby upstream. In the meantime, you can trivially add support in your own Dockerfile without recompiling ruby through the magic of LD_PRELOAD:

FROM ruby:2.6
RUN apt-get update && apt-get install libjemalloc1 && rm -rf /var/lib/apt/lists/*
ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1

# gleaned from a few sources
# https://github.com/jemalloc/jemalloc/wiki/Getting-Started
# https://brandonhilkert.com/blog/reducing-sidekiq-memory-usage-with-jemalloc/

And done, ruby now uses jemalloc. 🎉 🌮

[ftuyama]

@yosifkit I tried using LD_PRELOAD, but I have no clue why jemalloc isn't being loaded:

FROM ruby:2.4.2

RUN apt-get update && apt-get install -y libjemalloc-dev libjemalloc1
ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1

...

Then, inside the container:

> echo $LD_PRELOAD
/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
> ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']"
-lpthread -ldl -lcrypt -lm

Any idea why jemalloc isn't listed? 😿

Edit: ruby:2.6 won't work either.

[tjwallace]

@ftuyama in Ruby 2.6 I had to look for jemalloc in RbConfig::CONFIG['MAINLIBS'] (this is on my laptop, not in docker - we use the alpine image and there is an issue with jemalloc/musl right now)

[cesar-faria]

I am having trouble with jemalloc not being loaded as well.

@tjwallace I am using Ruby 2.6 and I got nothing in RbConfig::CONFIG['MAINLIBS']. 😢

[ftuyama]

Using a docker image where ruby is compiled using jemalloc, jemalloc is actually in MAINLIBS: https://hub.docker.com/r/swipesense/ruby-jemalloc

> ruby -r rbconfig -e "puts RbConfig::CONFIG['MAINLIBS']"
-lz -lpthread -lrt -lrt -ljemalloc -ldl -lcrypt -lm 

But I couldn't load it with LD_PRELOAD.

[tianon]

The whole point of LD_PRELOAD would be that Ruby isn't even aware it's being used -- it should be completely transparent to Ruby.

[butsjoh]

Normally when you do the following (when you see output) ruby should be using jemalloc.

MALLOC_CONF=stats_print:true ruby -e "exit"