Lua scripting in the rados gateway - ceph-users

25 Apr 2023

Hi ceph users,

I've been trying out the lua scripting for the rados gateway (thanks Yuval).

As in my previous email I mentioned that there is an error when trying to
load the luasocket module. However, I thought it was a good time to report
on my progress.

My 'hello world' example below is called *test.lua* below includes the
following checks:

   1. Can I write to the debug log?
   2. Can I use the lua socket package to do something stupid but
   intersting, like connect to a webservice?

Before you continue reading this, you might need to know that I run all
ceph processes in a *CentOS Stream release 8 *container deployed using ceph
orchestrator running *Ceph v17.2.5*, so please view the information below
in that context.

For anyone looking for a reference, I suggest going to the ceph lua rados
gateway documentation at radosgw/lua-scripting
<https://docs.ceph.com/en/quincy/radosgw/lua-scripting/>.

There are two new switches you need to know about in the radosgw-admin:

   - *script* -> loading your lua script
   - *script-package* -> loading supporting packages for your script - e.i.
   luasocket in this case.

For a basic setup, you'll need to have a few dependencies in your
containers:

   - cephadm container: requires luarocks (I've checked the code - it runs
   a luarocks search command)
   - radosgw container: requires luarocks, gcc, make,  m4, wget (wget just
   in case).

To achieve the above, I updated the container image for our running system.
I needed to do this because I needed to redeploy the rados gateway
container to inject the lua script packages into the radosgw runtime
process. This will start with a fresh container based on the global config
*container_image* setting on your running system.

For us this is currently captured in *quay.io/tsolo/ceph:v17.2.5-3
<http://quay.io/tsolo/ceph:v17.2.5-3>* and included the following exta
steps (including installing the lua dev from an rpm because there is no
centos package in yum):
yum install luarocks gcc make wget m4
rpm -i
https://rpmfind.net/linux/centos/8-stream/PowerTools/x86_64/os/Packages/lua…

You will notice that I've included a compiler and compiler support into the
image. This is because luarocks on the radosgw to compile luasocket (the
package I want to install). This will happen at start time when the radosgw
is restarted from ceph orch.

In the cephadm container I still need to update our cephadm shell so I need
to install luarocks by hand:
yum install luarocks

Then set thew updated image to use:
ceph config set global container_image quay.io/tsolo/ceph:v17.2.5-3

I now create a file called: *test.lua* in the cephadm container. This
contains the following lines to write to the log and then do a get request
to google. This is not practical in production, but it serves the purpose
of testing the infrastructure:

RGWDebugLog("Tsolo start lua script")
local LuaSocket = require("socket")
client = LuaSocket.connect("google.com", 80)
client:send("GET / HTTP/1.0\r\nHost: google.com\r\n\r\n")
while true do
  s, status, partial = client:receive('*a')
  RGWDebugLog(s or partial)
  if status == "closed" then
    break
  end
end
client:close()
RGWDebugLog("Tsolo stop lua")

Next I run:
radosgw-admin script-package add --package=luasocket --allow-compilation

And then list the added package to make sure it is there:
radosgw-admin script-package list

Note - at this point the radosgw has not been modified, it must first be
restarted.

Then I put the *test.lua *script into the pre request context:
radosgw-admin script put --infile=test.lua --context=preRequest

You also need to raise the debug log level on the running rados gateway:
ceph daemon
/var/run/ceph/ceph-client.rgw.xxx.xxx-cms1.xxxxx.x.xxxxxxxxxxxxxx.asok
config set debug_rgw 20

Inside the radosgw container I apply my fix (as per previous email):
cp -ru /tmp/luarocks/client.rgw.xxxxxx.xxx-xxxx-xxxx.pcoulb/lib64/*
/tmp/luarocks/client.rgw.xxxxxx.xxx-xxxx-xxxx.pcoulb/lib/

Outside on the host running the radosgw-admin container I follow the
journalctl for the radosgw container (to get the logs):
journalctl -fu ceph-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx@rgw.
xxx.xxx-cms1.xxxxx.x.xxxxxxxxxxxxxx.service

Then I run an s3cmd to put data in via the rados gateway and check the
journalctl logs and see:
Apr 25 20:54:47 brp-ceph-cms1 radosgw[60901]: Lua INFO: Tsolo start lua
Apr 25 20:54:47 brp-ceph-cms1 radosgw[60901]: Lua INFO: HTTP/1.0 301 Moved
Permanently
Apr 25 20:54:47 brp-ceph-cms1 radosgw[60901]: Lua INFO:
Apr 25 20:54:47 brp-ceph-cms1 radosgw[60901]: Lua INFO: Tsolo stop lua
Apr 25 20:54:47 brp-ceph-cms1 radosgw[60901]: Lua INFO: Tsolo start lua
Apr 25 20:54:48 brp-ceph-cms1 radosgw[60901]: Lua INFO: HTTP/1.0 301 Moved
Permanently
Apr 25 20:54:48 brp-ceph-cms1 radosgw[60901]: Lua INFO:
Apr 25 20:54:48 brp-ceph-cms1 radosgw[60901]: Lua INFO: Tsolo stop lua

So the script worked :)

If you want to see where the luarocks libraries have been installed,  look
in the /tmp/ directory of the radosgw container after you redeploy it and
you will find the content in /tmp/luarocks.

Conclusions:
There was a bit to figure out to get this working, but now that I've got
this simple test working I think there is a lot more to look into and
discover and use w.r.t. this powerful tool.

Cheers,
Tom