Hi Seena, The errno -5 is EIO, but as you saw it converts to 500--which is retryable (clients should re-try). As Yehuda said, the client could have stalled, but this error will also be generated if the client disconnects (e.g., timed out because RGW took too long). The logs should give a better indication what actually happened. Matt On Mon, Mar 23, 2020 at 9:00 AM Seena Fallah <seenafallah@gmail.com> wrote:
If it’s because of client why the status code changed to 500 (server error)? Isn’t it a client error?
On Mon, Mar 23, 2020 at 5:26 PM Yehuda Sadeh-Weinraub <yehuda@redhat.com> wrote:
On Mon, Mar 23, 2020 at 2:28 PM Seena Fallah <seenafallah@gmail.com> wrote:
Hi all. I often get this error on my PUT requests.
rgw.log-552376-2020-03-23 15:12:53.270 7fe5aa6da700 0 WARNING: set_req_state_err err_no=5 resorting to 500 rgw.log-552377-2020-03-23 15:12:53.270 7fe5aa6da700 1 ====== req done req=0x7fe5aa6d38c0 op status=-5 http_status=500 latency=59.9965s ======
My put requests are always under 1s but this requests spend 1min! Any logs I can read to find out what's going on with my cluster?
Likely that rgw waits for more data from the client and eventually times out. This can happen due to various reasons. You can bump up rgw logs (debug rgw = 20), and try to look at what the client thinks it's sending vs what rgw receives. You can also try to look at tcp capture (via wireshark for example).
Yehuda
_______________________________________________ Dev mailing list -- dev@ceph.io To unsubscribe send an email to dev-leave@ceph.io
-- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309