17 Feb
2023
17 Feb
'23
10:19 a.m.
Hi, please see the output below.
ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain is the one who is being
messed up with a wrong hostname. I want to delete it.
/iscsi-target...-igw/gateways> ls
o- gateways
..................................................................................................
[Up: 2/3, Portals: 3]
o- ceph-iscsi-gw-1.ipa.pthl.hk
.............................................................................
[172.16.202.251 (UP)]
o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
.............................................. [172.16.202.251
(UNAUTHORIZED)]
o- ceph-iscsi-gw-2.ipa.pthl.hk
.............................................................................
[172.16.202.252 (UP)]
/iscsi-target...-igw/gateways> delete
gateway_name=ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
confirm=true
Deleting gateway, ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
Could not contact ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain. If
the gateway is permanently down. Use confirm=true to force removal.
WARNING: Forcing removal of a gateway that can still be reached by an
initiator may result in data corruption.
/iscsi-target...-igw/gateways>
/iscsi-target...-igw/gateways> delete
gateway_name=ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
confirm=true
Deleting gateway, ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
Failed : Unhandled exception: list.remove(x): x not in list
However ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain is still there.
Version info is ceph-iscsi-3.5-1.el8cp.noarch on RHEL 8.4.
/iscsi-target...-igw/gateways> ls
o- gateways
..................................................................................................
[Up: 2/3, Portals: 3]
o- ceph-iscsi-gw-1.ipa.pthl.hk
.............................................................................
[172.16.202.251 (UP)]
o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
................................................... [172.16.202.251
(UNKNOWN)]
o- ceph-iscsi-gw-2.ipa.pthl.hk
.............................................................................
[172.16.202.252 (UP)]
/iscsi-target...-igw/gateways> delete
ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain confirm=true
Deleting gateway, ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
Failed : Unhandled exception: list.remove(x): x not in list
However ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain is still there.
Please help, thanks.
19 Feb
19 Feb
7:27 a.m.
On 17/02/2023 15:19, luckydog xf wrote:
Hi, please see the output below.
ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain is the one who is being
messed up with a wrong hostname. I want to delete it.
/iscsi-target...-igw/gateways> ls
o- gateways
..................................................................................................
[Up: 2/3, Portals: 3]
o- ceph-iscsi-gw-1.ipa.pthl.hk
.............................................................................
[172.16.202.251 (UP)]
o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
.............................................. [172.16.202.251
(UNAUTHORIZED)]
o- ceph-iscsi-gw-2.ipa.pthl.hk
.............................................................................
[172.16.202.252 (UP)]
/iscsi-target...-igw/gateways> delete
gateway_name=ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
confirm=true
Deleting gateway, ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
Could not contact ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain. If
the gateway is permanently down. Use confirm=true to force removal.
WARNING: Forcing removal of a gateway that can still be reached by an
initiator may result in data corruption.
/iscsi-target...-igw/gateways>
/iscsi-target...-igw/gateways> delete
gateway_name=ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
confirm=true
Deleting gateway, ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
Failed : Unhandled exception: list.remove(x): x not in list
However ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain is still there.
Version info is ceph-iscsi-3.5-1.el8cp.noarch on RHEL 8.4.
/iscsi-target...-igw/gateways> ls
o- gateways
..................................................................................................
[Up: 2/3, Portals: 3]
o- ceph-iscsi-gw-1.ipa.pthl.hk
.............................................................................
[172.16.202.251 (UP)]
o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
................................................... [172.16.202.251
(UNKNOWN)]
o- ceph-iscsi-gw-2.ipa.pthl.hk
.............................................................................
[172.16.202.252 (UP)]
/iscsi-target...-igw/gateways> delete
ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain confirm=true
Deleting gateway, ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
Failed : Unhandled exception: list.remove(x): x not in list
However ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain is still there.
Hi
It seems you have corrupted the iscsi-gateway.cfg contents.
Could you share the following info ?
1, `gwcli ls` output
2, iscsi-gateway.cfg object's contents.
Thanks,
Please help, thanks.
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
--
Best Regards,
Xiubo Li (李秀波)
Email: xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
20 Feb
20 Feb
4:20 a.m.
Hi,
[root@ceph-iscsi-gw-1 ~]# gwcli ls
o- /
.........................................................................................................................
[...]
o- cluster
.........................................................................................................
[Clusters: 1]
| o- ceph
............................................................................................................
[HEALTH_OK]
| o- pools
.........................................................................................................
[Pools: 22]
| | o- .rgw.root
.............................................................. [(x3),
Commit: 0.00Y/115222232M (0%), Used: 52K]
| | o- cinder-ceph ................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 28703946995700b]
| | o- default.rgw.buckets.data
............................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 4587b]
| | o- default.rgw.buckets.extra
............................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.buckets.index
............................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.control
.................................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.data.root
................................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.gc
....................................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.intent-log
............................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.log
....................................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 384K]
| | o- default.rgw.meta
....................................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 24K]
| | o- default.rgw.usage
.................................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.email
.............................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.keys
............................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.swift
.............................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.uid
................................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- device_health_metrics
........................................... [(x3), Commit: 0.00Y/115222232M
(0%), Used: 305303172b]
| | o- glance .......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 525074107902b]
| | o- gnocchi
......................................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 989498938b]
| | o- nova
................................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 12K]
| | o- rbd ........................................................
[(x3), Commit: 20.0T/115222232M (18%), Used: 3274867542037b]
| | o- scbench
.......................................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 88829964K]
| o- topology
..............................................................................................
[OSDs: 108,MONs: 3]
o- disks
.......................................................................................................
[20.0T, Disks: 1]
| o- rbd
...........................................................................................................
[rbd (20.0T)]
| o- ceph-iscsi
...............................................................................
[rbd/ceph-iscsi (Online, 20.0T)]
o- iscsi-targets
...............................................................................
[DiscoveryAuth: None, Targets: 1]
o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
......................................................... [Auth: None,
Gateways: 3]
o- disks
..........................................................................................................
[Disks: 1]
| o- rbd/ceph-iscsi
............................................................. [Owner:
ceph-iscsi-gw-1.ipa.pthl.hk, Lun: 0]
o- gateways
............................................................................................
[Up: 2/3, Portals: 3]
| o- ceph-iscsi-gw-1.ipa.pthl.hk
.......................................................................
[172.16.202.251 (UP)]
| o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
............................................. [172.16.202.251 (UNKNOWN)]
| o- ceph-iscsi-gw-2.ipa.pthl.hk
.......................................................................
[172.16.202.252 (UP)]
o- host-groups
..................................................................................................
[Groups : 0]
o- hosts
......................................................................................
[Auth: ACL_DISABLED, Hosts: 0]
===
[root@ceph-iscsi-gw-1 ~]# cat /etc/ceph/iscsi-gateway.cfg
[config]
# Name of the Ceph storage cluster. A suitable Ceph configuration file
allowing
# access to the Ceph storage cluster from the gateway node is required, if
not
# colocated on an OSD node.
cluster_name = ceph
# Place a copy of the ceph cluster's admin keyring in the gateway's
/etc/ceph
# drectory and reference the filename here
gateway_keyring = ceph.client.admin.keyring
# API settings.
# The API supports a number of options that allow you to tailor it to your
# local environment. If you want to run the API under https, you will need
to
# create cert/key files that are compatible for each iSCSI gateway node,
that is
# not locked to a specific node. SSL cert and key files *must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the
'/etc/ceph/' directory
# on *each* gateway node. With the SSL files in place, you can use
'api_secure = true'
# to switch to https mode.
# To support the API, the bare minimum settings are:
api_secure = false
# Additional API configuration options are as follows, defaults shown.
api_user = admin
api_password = admin
api_port = 5001
# API IP
trusted_ip_list = 172.16.200.251,172.16.200.252
4:30 a.m.
Hi
So you are using the default 'rbd' pool to store 'gateway.conf' config
object.
And the 'gateway.conf' is corrupted. If you couldn't remove it by using
force option, then you should dump the 'gateway.conf' object from 'rbd'
pool and then delete the
'ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain' related configures
manually and then store it back. Then to see will it disappear, if not
please restart the ceph-iscsi services.
If you are not sure what should be deleted you can send it to me, I will
help revise it for you.
Thanks
- Xiubo
On 20/02/2023 09:20, luckydog xf wrote:
Hi,
[root@ceph-iscsi-gw-1 ~]# gwcli ls
o- /
.........................................................................................................................
[...]
o- cluster
.........................................................................................................
[Clusters: 1]
| o- ceph
............................................................................................................
[HEALTH_OK]
| o- pools
.........................................................................................................
[Pools: 22]
| | o- .rgw.root
..............................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 52K]
| | o- cinder-ceph
................................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 28703946995700b]
| | o- default.rgw.buckets.data
............................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 4587b]
| | o- default.rgw.buckets.extra
............................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.buckets.index
............................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.control
.................................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.data.root
................................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.gc
....................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.intent-log
............................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.log
....................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 384K]
| | o- default.rgw.meta
....................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 24K]
| | o- default.rgw.usage
.................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.email
.............................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.keys
............................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.swift
.............................................. [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.uid
................................................ [(x3), Commit:
0.00Y/115222232M (0%), Used: 0.00Y]
| | o- device_health_metrics
........................................... [(x3), Commit:
0.00Y/115222232M (0%), Used: 305303172b]
| | o- glance
....................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 525074107902b]
| | o- gnocchi
......................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 989498938b]
| | o- nova
...................................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 12K]
| | o- rbd
........................................................ [(x3),
Commit: 20.0T/115222232M (18%), Used: 3274867542037b]
| | o- scbench
.......................................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 88829964K]
| o- topology
..............................................................................................
[OSDs: 108,MONs: 3]
o- disks
.......................................................................................................
[20.0T, Disks: 1]
| o- rbd
...........................................................................................................
[rbd (20.0T)]
| o- ceph-iscsi
...............................................................................
[rbd/ceph-iscsi (Online, 20.0T)]
o- iscsi-targets
...............................................................................
[DiscoveryAuth: None, Targets: 1]
o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
......................................................... [Auth:
None, Gateways: 3]
o- disks
..........................................................................................................
[Disks: 1]
| o- rbd/ceph-iscsi
.............................................................
[Owner: ceph-iscsi-gw-1.ipa.pthl.hk
<http://ceph-iscsi-gw-1.ipa.pthl.hk>, Lun: 0]
o- gateways
............................................................................................
[Up: 2/3, Portals: 3]
| o- ceph-iscsi-gw-1.ipa.pthl.hk
<http://ceph-iscsi-gw-1.ipa.pthl.hk>
.......................................................................
[172.16.202.251 (UP)]
| o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
............................................. [172.16.202.251
(UNKNOWN)]
| o- ceph-iscsi-gw-2.ipa.pthl.hk
<http://ceph-iscsi-gw-2.ipa.pthl.hk>
.......................................................................
[172.16.202.252 (UP)]
o- host-groups
..................................................................................................
[Groups : 0]
o- hosts
......................................................................................
[Auth: ACL_DISABLED, Hosts: 0]
===
[root@ceph-iscsi-gw-1 ~]# cat /etc/ceph/iscsi-gateway.cfg
[config]
# Name of the Ceph storage cluster. A suitable Ceph configuration file
allowing
# access to the Ceph storage cluster from the gateway node is
required, if not
# colocated on an OSD node.
cluster_name = ceph
# Place a copy of the ceph cluster's admin keyring in the gateway's
/etc/ceph
# drectory and reference the filename here
gateway_keyring = ceph.client.admin.keyring
# API settings.
# The API supports a number of options that allow you to tailor it to your
# local environment. If you want to run the API under https, you will
need to
# create cert/key files that are compatible for each iSCSI gateway
node, that is
# not locked to a specific node. SSL cert and key files *must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the
'/etc/ceph/' directory
# on *each* gateway node. With the SSL files in place, you can use
'api_secure = true'
# to switch to https mode.
# To support the API, the bare minimum settings are:
api_secure = false
# Additional API configuration options are as follows, defaults shown.
api_user = admin
api_password = admin
api_port = 5001
# API IP
trusted_ip_list = 172.16.200.251,172.16.200.252
--
Best Regards,
Xiubo Li (李秀波)
Email:xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
4:50 a.m.
Hi, Xiubo.
I'm not clear about the commands that dump and restore an object. Could you
give me an example?
`rados ls -p rbd` shows tons of uuids.
https://docs.ceph.com/en/latest/man/8/rados/
On Mon, Feb 20, 2023 at 9:30 AM Xiubo Li <xiubli(a)redhat.com> wrote:
Hi
So you are using the default 'rbd' pool to store 'gateway.conf' config
object.
And the 'gateway.conf' is corrupted. If you couldn't remove it by using
force option, then you should dump the 'gateway.conf' object from 'rbd'
pool and then delete the 'ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain'
related configures manually and then store it back. Then to see will it
disappear, if not please restart the ceph-iscsi services.
If you are not sure what should be deleted you can send it to me, I will
help revise it for you.
Thanks
- Xiubo
On 20/02/2023 09:20, luckydog xf wrote:
Hi,
> [root@ceph-iscsi-gw-1 ~]# gwcli ls
> o- /
>
.........................................................................................................................
> [...]
> o- cluster
>
.........................................................................................................
> [Clusters: 1]
> | o- ceph
>
............................................................................................................
> [HEALTH_OK]
> | o- pools
>
.........................................................................................................
> [Pools: 22]
> | | o- .rgw.root
> .............................................................. [(x3),
> Commit: 0.00Y/115222232M (0%), Used: 52K]
> | | o- cinder-ceph ................................................
> [(x3), Commit: 0.00Y/115222232M (0%), Used: 28703946995700b]
> | | o- default.rgw.buckets.data
> ............................................. [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 4587b]
> | | o- default.rgw.buckets.extra
> ............................................ [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.buckets.index
> ............................................ [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.control
> .................................................. [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.data.root
> ................................................ [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.gc
> ....................................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.intent-log
> ............................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.log
> ....................................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 384K]
> | | o- default.rgw.meta
> ....................................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 24K]
> | | o- default.rgw.usage
> .................................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.users.email
> .............................................. [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.users.keys
> ............................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.users.swift
> .............................................. [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- default.rgw.users.uid
> ................................................ [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 0.00Y]
> | | o- device_health_metrics
> ........................................... [(x3), Commit: 0.00Y/115222232M
> (0%), Used: 305303172b]
> | | o- glance .......................................................
> [(x3), Commit: 0.00Y/115222232M (0%), Used: 525074107902b]
> | | o- gnocchi
> ......................................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 989498938b]
> | | o- nova
> ................................................................... [(x3),
> Commit: 0.00Y/115222232M (0%), Used: 12K]
> | | o- rbd ........................................................
> [(x3), Commit: 20.0T/115222232M (18%), Used: 3274867542037b]
> | | o- scbench
> .......................................................... [(x3), Commit:
> 0.00Y/115222232M (0%), Used: 88829964K]
> | o- topology
>
..............................................................................................
> [OSDs: 108,MONs: 3]
> o- disks
>
.......................................................................................................
> [20.0T, Disks: 1]
> | o- rbd
>
...........................................................................................................
> [rbd (20.0T)]
> | o- ceph-iscsi
> ...............................................................................
> [rbd/ceph-iscsi (Online, 20.0T)]
> o- iscsi-targets
> ...............................................................................
> [DiscoveryAuth: None, Targets: 1]
> o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
> ......................................................... [Auth: None,
> Gateways: 3]
> o- disks
>
..........................................................................................................
> [Disks: 1]
> | o- rbd/ceph-iscsi
> ............................................................. [Owner:
> ceph-iscsi-gw-1.ipa.pthl.hk, Lun: 0]
> o- gateways
>
............................................................................................
> [Up: 2/3, Portals: 3]
> | o- ceph-iscsi-gw-1.ipa.pthl.hk
> .......................................................................
> [172.16.202.251 (UP)]
> | o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
> ............................................. [172.16.202.251 (UNKNOWN)]
> | o- ceph-iscsi-gw-2.ipa.pthl.hk
> .......................................................................
> [172.16.202.252 (UP)]
> o- host-groups
>
..................................................................................................
> [Groups : 0]
> o- hosts
>
......................................................................................
> [Auth: ACL_DISABLED, Hosts: 0]
===
[root@ceph-iscsi-gw-1 ~]# cat /etc/ceph/iscsi-gateway.cfg
[config]
# Name of the Ceph storage cluster. A suitable Ceph configuration file
allowing
# access to the Ceph storage cluster from the gateway node is required, if
not
# colocated on an OSD node.
cluster_name = ceph
# Place a copy of the ceph cluster's admin keyring in the gateway's
/etc/ceph
# drectory and reference the filename here
gateway_keyring = ceph.client.admin.keyring
# API settings.
# The API supports a number of options that allow you to tailor it to your
# local environment. If you want to run the API under https, you will need
to
# create cert/key files that are compatible for each iSCSI gateway node,
that is
# not locked to a specific node. SSL cert and key files *must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the
'/etc/ceph/' directory
# on *each* gateway node. With the SSL files in place, you can use
'api_secure = true'
# to switch to https mode.
# To support the API, the bare minimum settings are:
api_secure = false
# Additional API configuration options are as follows, defaults shown.
api_user = admin
api_password = admin
api_port = 5001
# API IP
trusted_ip_list = 172.16.200.251,172.16.200.252
--
Best Regards,
Xiubo Li (李秀波)
Email: xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
5:11 a.m.
Oh, please forgive my carelessness.
--
sudo rados -p rbd get gateway.conf local-gw.conf
---
get all the contents. I'll try to edit it.
On Mon, Feb 20, 2023 at 9:50 AM luckydog xf <luckydogxf(a)gmail.com> wrote:
Hi, Xiubo.
I'm not clear about the commands that dump and restore an object. Could
you give me an example?
`rados ls -p rbd` shows tons of uuids.
https://docs.ceph.com/en/latest/man/8/rados/
On Mon, Feb 20, 2023 at 9:30 AM Xiubo Li <xiubli(a)redhat.com> wrote:
> Hi
>
> So you are using the default 'rbd' pool to store 'gateway.conf'
config
> object.
>
> And the 'gateway.conf' is corrupted. If you couldn't remove it by using
> force option, then you should dump the 'gateway.conf' object from
'rbd'
> pool and then delete the 'ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain'
> related configures manually and then store it back. Then to see will it
> disappear, if not please restart the ceph-iscsi services.
>
> If you are not sure what should be deleted you can send it to me, I will
> help revise it for you.
>
> Thanks
>
> - Xiubo
> On 20/02/2023 09:20, luckydog xf wrote:
>
> Hi,
>>
>
>
>> [root@ceph-iscsi-gw-1 ~]# gwcli ls
>> o- /
>>
.........................................................................................................................
>> [...]
>> o- cluster
>>
.........................................................................................................
>> [Clusters: 1]
>> | o- ceph
>>
............................................................................................................
>> [HEALTH_OK]
>> | o- pools
>>
.........................................................................................................
>> [Pools: 22]
>> | | o- .rgw.root
>> .............................................................. [(x3),
>> Commit: 0.00Y/115222232M (0%), Used: 52K]
>> | | o- cinder-ceph ................................................
>> [(x3), Commit: 0.00Y/115222232M (0%), Used: 28703946995700b]
>> | | o- default.rgw.buckets.data
>> ............................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 4587b]
>> | | o- default.rgw.buckets.extra
>> ............................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.buckets.index
>> ............................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.control
>> .................................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.data.root
>> ................................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.gc
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.intent-log
>> ............................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.log
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 384K]
>> | | o- default.rgw.meta
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 24K]
>> | | o- default.rgw.usage
>> .................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.email
>> .............................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.keys
>> ............................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.swift
>> .............................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.uid
>> ................................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- device_health_metrics
>> ........................................... [(x3), Commit: 0.00Y/115222232M
>> (0%), Used: 305303172b]
>> | | o- glance
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 525074107902b]
>> | | o- gnocchi
>> ......................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 989498938b]
>> | | o- nova
>> ................................................................... [(x3),
>> Commit: 0.00Y/115222232M (0%), Used: 12K]
>> | | o- rbd ........................................................
>> [(x3), Commit: 20.0T/115222232M (18%), Used: 3274867542037b]
>> | | o- scbench
>> .......................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 88829964K]
>> | o- topology
>>
..............................................................................................
>> [OSDs: 108,MONs: 3]
>> o- disks
>>
.......................................................................................................
>> [20.0T, Disks: 1]
>> | o- rbd
>>
...........................................................................................................
>> [rbd (20.0T)]
>> | o- ceph-iscsi
>> ...............................................................................
>> [rbd/ceph-iscsi (Online, 20.0T)]
>> o- iscsi-targets
>> ...............................................................................
>> [DiscoveryAuth: None, Targets: 1]
>> o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
>> ......................................................... [Auth: None,
>> Gateways: 3]
>> o- disks
>>
..........................................................................................................
>> [Disks: 1]
>> | o- rbd/ceph-iscsi
>> ............................................................. [Owner:
>> ceph-iscsi-gw-1.ipa.pthl.hk, Lun: 0]
>> o- gateways
>>
............................................................................................
>> [Up: 2/3, Portals: 3]
>> | o- ceph-iscsi-gw-1.ipa.pthl.hk
>> .......................................................................
>> [172.16.202.251 (UP)]
>> | o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
>> ............................................. [172.16.202.251 (UNKNOWN)]
>> | o- ceph-iscsi-gw-2.ipa.pthl.hk
>> .......................................................................
>> [172.16.202.252 (UP)]
>> o- host-groups
>>
..................................................................................................
>> [Groups : 0]
>> o- hosts
>>
......................................................................................
>> [Auth: ACL_DISABLED, Hosts: 0]
>>
>
>
> ===
> [root@ceph-iscsi-gw-1 ~]# cat /etc/ceph/iscsi-gateway.cfg
> [config]
> # Name of the Ceph storage cluster. A suitable Ceph configuration file
> allowing
> # access to the Ceph storage cluster from the gateway node is required,
> if not
> # colocated on an OSD node.
> cluster_name = ceph
>
> # Place a copy of the ceph cluster's admin keyring in the gateway's
> /etc/ceph
> # drectory and reference the filename here
> gateway_keyring = ceph.client.admin.keyring
>
>
> # API settings.
> # The API supports a number of options that allow you to tailor it to your
> # local environment. If you want to run the API under https, you will
> need to
> # create cert/key files that are compatible for each iSCSI gateway node,
> that is
> # not locked to a specific node. SSL cert and key files *must* be called
> # 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the
> '/etc/ceph/' directory
> # on *each* gateway node. With the SSL files in place, you can use
> 'api_secure = true'
> # to switch to https mode.
>
> # To support the API, the bare minimum settings are:
> api_secure = false
>
> # Additional API configuration options are as follows, defaults shown.
> api_user = admin
> api_password = admin
> api_port = 5001
> # API IP
> trusted_ip_list = 172.16.200.251,172.16.200.252
>
>
> --
> Best Regards,
>
> Xiubo Li (李秀波)
>
> Email: xiubli@redhat.com/xiubli@ibm.com
> Slack: @Xiubo Li
>
>
5:13 a.m.
On 20/02/2023 10:11, luckydog xf wrote:
Oh, please forgive my carelessness.
--
sudo rados -p rbd get gateway.conf local-gw.conf
---
get all the contents. I'll try to edit it.
Correct.
On Mon, Feb 20, 2023 at 9:50 AM luckydog xf
<luckydogxf(a)gmail.com> wrote:
Hi, Xiubo.
I'm not clear about the commands that dump and restore an object.
Could you give me an example?
`rados ls -p rbd` shows tons of uuids.
https://docs.ceph.com/en/latest/man/8/rados/
On Mon, Feb 20, 2023 at 9:30 AM Xiubo Li <xiubli(a)redhat.com> wrote:
Hi
So you are using the default 'rbd' pool to store
'gateway.conf' config object.
And the 'gateway.conf' is corrupted. If you couldn't remove it
by using force option, then you should dump the 'gateway.conf'
object from 'rbd' pool and then delete the
'ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain' related
configures manually and then store it back. Then to see will
it disappear, if not please restart the ceph-iscsi services.
If you are not sure what should be deleted you can send it to
me, I will help revise it for you.
Thanks
- Xiubo
On 20/02/2023 09:20, luckydog xf wrote:
--
Best Regards,
Xiubo Li (李秀波)
Email:xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
Hi,
[root@ceph-iscsi-gw-1 ~]# gwcli ls
o- /
.........................................................................................................................
[...]
o- cluster
.........................................................................................................
[Clusters: 1]
| o- ceph
............................................................................................................
[HEALTH_OK]
| o- pools
.........................................................................................................
[Pools: 22]
| | o- .rgw.root
..............................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 52K]
| | o- cinder-ceph
................................................ [(x3),
Commit: 0.00Y/115222232M (0%), Used: 28703946995700b]
| | o- default.rgw.buckets.data
............................................. [(x3),
Commit: 0.00Y/115222232M (0%), Used: 4587b]
| | o- default.rgw.buckets.extra
............................................ [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.buckets.index
............................................ [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.control
.................................................. [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.data.root
................................................ [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.gc
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.intent-log
............................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.log
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 384K]
| | o- default.rgw.meta
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 24K]
| | o- default.rgw.usage
....................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.email
.............................................. [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.keys
............................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.swift
.............................................. [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.uid
................................................ [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- device_health_metrics
........................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 305303172b]
| | o- glance
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 525074107902b]
| | o- gnocchi
.........................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 989498938b]
| | o- nova
...................................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 12K]
| | o- rbd
........................................................
[(x3), Commit: 20.0T/115222232M (18%), Used: 3274867542037b]
| | o- scbench
..........................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 88829964K]
| o- topology
..............................................................................................
[OSDs: 108,MONs: 3]
o- disks
.......................................................................................................
[20.0T, Disks: 1]
| o- rbd
...........................................................................................................
[rbd (20.0T)]
| o- ceph-iscsi
...............................................................................
[rbd/ceph-iscsi (Online, 20.0T)]
o- iscsi-targets
...............................................................................
[DiscoveryAuth: None, Targets: 1]
o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
.........................................................
[Auth: None, Gateways: 3]
o- disks
..........................................................................................................
[Disks: 1]
| o- rbd/ceph-iscsi
.............................................................
[Owner: ceph-iscsi-gw-1.ipa.pthl.hk
<http://ceph-iscsi-gw-1.ipa.pthl.hk>, Lun: 0]
o- gateways
............................................................................................
[Up: 2/3, Portals: 3]
| o- ceph-iscsi-gw-1.ipa.pthl.hk
<http://ceph-iscsi-gw-1.ipa.pthl.hk>
.......................................................................
[172.16.202.251 (UP)]
| o-
ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
.............................................
[172.16.202.251 (UNKNOWN)]
| o- ceph-iscsi-gw-2.ipa.pthl.hk
<http://ceph-iscsi-gw-2.ipa.pthl.hk>
.......................................................................
[172.16.202.252 (UP)]
o- host-groups
..................................................................................................
[Groups : 0]
o- hosts
......................................................................................
[Auth: ACL_DISABLED, Hosts: 0]
===
[root@ceph-iscsi-gw-1 ~]# cat /etc/ceph/iscsi-gateway.cfg
[config]
# Name of the Ceph storage cluster. A suitable Ceph
configuration file allowing
# access to the Ceph storage cluster from the gateway node is
required, if not
# colocated on an OSD node.
cluster_name = ceph
# Place a copy of the ceph cluster's admin keyring in the
gateway's /etc/ceph
# drectory and reference the filename here
gateway_keyring = ceph.client.admin.keyring
# API settings.
# The API supports a number of options that allow you to
tailor it to your
# local environment. If you want to run the API under https,
you will need to
# create cert/key files that are compatible for each iSCSI
gateway node, that is
# not locked to a specific node. SSL cert and key files
*must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in
the '/etc/ceph/' directory
# on *each* gateway node. With the SSL files in place, you
can use 'api_secure = true'
# to switch to https mode.
# To support the API, the bare minimum settings are:
api_secure = false
# Additional API configuration options are as follows,
defaults shown.
api_user = admin
api_password = admin
api_port = 5001
# API IP
trusted_ip_list = 172.16.200.251,172.16.200.252
--
Best Regards,
Xiubo Li (李秀波)
Email:xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
5:19 a.m.
okay, I restored the correct configuration by 'sudo rados put gateway.conf
local-gw -p rbd
Now problems resolved. Thanks and have a nice day.
On Mon, Feb 20, 2023 at 10:13 AM Xiubo Li <xiubli(a)redhat.com> wrote:
On 20/02/2023 10:11, luckydog xf wrote:
Oh, please forgive my carelessness.
--
sudo rados -p rbd get gateway.conf local-gw.conf
---
get all the contents. I'll try to edit it.
Correct.
On Mon, Feb 20, 2023 at 9:50 AM luckydog xf <luckydogxf(a)gmail.com> wrote:
Hi, Xiubo.
I'm not clear about the commands that dump and restore an object. Could
you give me an example?
`rados ls -p rbd` shows tons of uuids.
https://docs.ceph.com/en/latest/man/8/rados/
On Mon, Feb 20, 2023 at 9:30 AM Xiubo Li <xiubli(a)redhat.com> wrote:
> Hi
>
> So you are using the default 'rbd' pool to store 'gateway.conf'
config
> object.
>
> And the 'gateway.conf' is corrupted. If you couldn't remove it by using
> force option, then you should dump the 'gateway.conf' object from
'rbd'
> pool and then delete the 'ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain'
> related configures manually and then store it back. Then to see will it
> disappear, if not please restart the ceph-iscsi services.
>
> If you are not sure what should be deleted you can send it to me, I will
> help revise it for you.
>
> Thanks
>
> - Xiubo
> On 20/02/2023 09:20, luckydog xf wrote:
>
> Hi,
>>
>
>
>> [root@ceph-iscsi-gw-1 ~]# gwcli ls
>> o- /
>>
.........................................................................................................................
>> [...]
>> o- cluster
>>
.........................................................................................................
>> [Clusters: 1]
>> | o- ceph
>>
............................................................................................................
>> [HEALTH_OK]
>> | o- pools
>>
.........................................................................................................
>> [Pools: 22]
>> | | o- .rgw.root
>> .............................................................. [(x3),
>> Commit: 0.00Y/115222232M (0%), Used: 52K]
>> | | o- cinder-ceph ................................................
>> [(x3), Commit: 0.00Y/115222232M (0%), Used: 28703946995700b]
>> | | o- default.rgw.buckets.data
>> ............................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 4587b]
>> | | o- default.rgw.buckets.extra
>> ............................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.buckets.index
>> ............................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.control
>> .................................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.data.root
>> ................................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.gc
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.intent-log
>> ............................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.log
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 384K]
>> | | o- default.rgw.meta
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 24K]
>> | | o- default.rgw.usage
>> .................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.email
>> .............................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.keys
>> ............................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.swift
>> .............................................. [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- default.rgw.users.uid
>> ................................................ [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 0.00Y]
>> | | o- device_health_metrics
>> ........................................... [(x3), Commit: 0.00Y/115222232M
>> (0%), Used: 305303172b]
>> | | o- glance
>> ....................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 525074107902b]
>> | | o- gnocchi
>> ......................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 989498938b]
>> | | o- nova
>> ................................................................... [(x3),
>> Commit: 0.00Y/115222232M (0%), Used: 12K]
>> | | o- rbd ........................................................
>> [(x3), Commit: 20.0T/115222232M (18%), Used: 3274867542037b]
>> | | o- scbench
>> .......................................................... [(x3), Commit:
>> 0.00Y/115222232M (0%), Used: 88829964K]
>> | o- topology
>>
..............................................................................................
>> [OSDs: 108,MONs: 3]
>> o- disks
>>
.......................................................................................................
>> [20.0T, Disks: 1]
>> | o- rbd
>>
...........................................................................................................
>> [rbd (20.0T)]
>> | o- ceph-iscsi
>> ...............................................................................
>> [rbd/ceph-iscsi (Online, 20.0T)]
>> o- iscsi-targets
>> ...............................................................................
>> [DiscoveryAuth: None, Targets: 1]
>> o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
>> ......................................................... [Auth: None,
>> Gateways: 3]
>> o- disks
>>
..........................................................................................................
>> [Disks: 1]
>> | o- rbd/ceph-iscsi
>> ............................................................. [Owner:
>> ceph-iscsi-gw-1.ipa.pthl.hk, Lun: 0]
>> o- gateways
>>
............................................................................................
>> [Up: 2/3, Portals: 3]
>> | o- ceph-iscsi-gw-1.ipa.pthl.hk
>> .......................................................................
>> [172.16.202.251 (UP)]
>> | o- ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
>> ............................................. [172.16.202.251 (UNKNOWN)]
>> | o- ceph-iscsi-gw-2.ipa.pthl.hk
>> .......................................................................
>> [172.16.202.252 (UP)]
>> o- host-groups
>>
..................................................................................................
>> [Groups : 0]
>> o- hosts
>>
......................................................................................
>> [Auth: ACL_DISABLED, Hosts: 0]
>>
>
>
> ===
> [root@ceph-iscsi-gw-1 ~]# cat /etc/ceph/iscsi-gateway.cfg
> [config]
> # Name of the Ceph storage cluster. A suitable Ceph configuration file
> allowing
> # access to the Ceph storage cluster from the gateway node is required,
> if not
> # colocated on an OSD node.
> cluster_name = ceph
>
> # Place a copy of the ceph cluster's admin keyring in the gateway's
> /etc/ceph
> # drectory and reference the filename here
> gateway_keyring = ceph.client.admin.keyring
>
>
> # API settings.
> # The API supports a number of options that allow you to tailor it to
> your
> # local environment. If you want to run the API under https, you will
> need to
> # create cert/key files that are compatible for each iSCSI gateway node,
> that is
> # not locked to a specific node. SSL cert and key files *must* be called
> # 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the
> '/etc/ceph/' directory
> # on *each* gateway node. With the SSL files in place, you can use
> 'api_secure = true'
> # to switch to https mode.
>
> # To support the API, the bare minimum settings are:
> api_secure = false
>
> # Additional API configuration options are as follows, defaults shown.
> api_user = admin
> api_password = admin
> api_port = 5001
> # API IP
> trusted_ip_list = 172.16.200.251,172.16.200.252
>
>
> --
> Best Regards,
>
> Xiubo Li (李秀波)
>
> Email: xiubli@redhat.com/xiubli@ibm.com
> Slack: @Xiubo Li
>
> --
Best Regards,
Xiubo Li (李秀波)
Email: xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
5:21 a.m.
Cool :-)
On 20/02/2023 10:19, luckydog xf wrote:
okay, I restored the correct configuration by
'sudo rados put
gateway.conf local-gw -p rbd
Now problems resolved. Thanks and have a nice day.
On Mon, Feb 20, 2023 at 10:13 AM Xiubo Li <xiubli(a)redhat.com> wrote:
On 20/02/2023 10:11, luckydog xf wrote:
--
Best Regards,
Xiubo Li (李秀波)
Email:xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
Oh, please forgive my carelessness.
--
sudo rados -p rbd get gateway.conf local-gw.conf
---
get all the contents. I'll try to edit it.
Correct.
On Mon, Feb 20, 2023 at 9:50 AM luckydog xf
<luckydogxf(a)gmail.com> wrote:
Hi, Xiubo.
I'm not clear about the commands that dump and restore an
object. Could you give me an example?
`rados ls -p rbd` shows tons of uuids.
https://docs.ceph.com/en/latest/man/8/rados/
On Mon, Feb 20, 2023 at 9:30 AM Xiubo Li <xiubli(a)redhat.com>
wrote:
Hi
So you are using the default 'rbd' pool to store
'gateway.conf' config object.
And the 'gateway.conf' is corrupted. If you couldn't
remove it by using force option, then you should dump the
'gateway.conf' object from 'rbd' pool and then delete the
'ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain'
related configures manually and then store it back. Then
to see will it disappear, if not please restart the
ceph-iscsi services.
If you are not sure what should be deleted you can send
it to me, I will help revise it for you.
Thanks
- Xiubo
On 20/02/2023 09:20, luckydog xf wrote:
--
Best Regards,
Xiubo Li (李秀波)
Email:xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
Hi,
[root@ceph-iscsi-gw-1 ~]# gwcli ls
o- /
.........................................................................................................................
[...]
o- cluster
.........................................................................................................
[Clusters: 1]
| o- ceph
............................................................................................................
[HEALTH_OK]
| o- pools
.........................................................................................................
[Pools: 22]
| | o- .rgw.root
..............................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 52K]
| | o- cinder-ceph
................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used:
28703946995700b]
| | o- default.rgw.buckets.data
............................................. [(x3),
Commit: 0.00Y/115222232M (0%), Used: 4587b]
| | o- default.rgw.buckets.extra
............................................ [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.buckets.index
............................................ [(x3),
Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.control
..................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.data.root
................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.gc
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.intent-log
...............................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.log
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 384K]
| | o- default.rgw.meta
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 24K]
| | o- default.rgw.usage
....................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.email
..............................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.keys
...............................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.swift
..............................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- default.rgw.users.uid
................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 0.00Y]
| | o- device_health_metrics
........................................... [(x3),
Commit: 0.00Y/115222232M (0%), Used: 305303172b]
| | o- glance
.......................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used:
525074107902b]
| | o- gnocchi
.........................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 989498938b]
| | o- nova
...................................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 12K]
| | o- rbd
........................................................
[(x3), Commit: 20.0T/115222232M (18%), Used:
3274867542037b]
| | o- scbench
..........................................................
[(x3), Commit: 0.00Y/115222232M (0%), Used: 88829964K]
| o- topology
..............................................................................................
[OSDs: 108,MONs: 3]
o- disks
.......................................................................................................
[20.0T, Disks: 1]
| o- rbd
...........................................................................................................
[rbd (20.0T)]
| o- ceph-iscsi
...............................................................................
[rbd/ceph-iscsi (Online, 20.0T)]
o- iscsi-targets
...............................................................................
[DiscoveryAuth: None, Targets: 1]
o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
.........................................................
[Auth: None, Gateways: 3]
o- disks
..........................................................................................................
[Disks: 1]
| o- rbd/ceph-iscsi
.............................................................
[Owner: ceph-iscsi-gw-1.ipa.pthl.hk
<http://ceph-iscsi-gw-1.ipa.pthl.hk>, Lun: 0]
o- gateways
............................................................................................
[Up: 2/3, Portals: 3]
| o- ceph-iscsi-gw-1.ipa.pthl.hk
<http://ceph-iscsi-gw-1.ipa.pthl.hk>
.......................................................................
[172.16.202.251 (UP)]
| o-
ceph-iscsi-gw-1.ipa.pthl.hklocalhost.localdomain
.............................................
[172.16.202.251 (UNKNOWN)]
| o- ceph-iscsi-gw-2.ipa.pthl.hk
<http://ceph-iscsi-gw-2.ipa.pthl.hk>
.......................................................................
[172.16.202.252 (UP)]
o- host-groups
..................................................................................................
[Groups : 0]
o- hosts
......................................................................................
[Auth: ACL_DISABLED, Hosts: 0]
===
[root@ceph-iscsi-gw-1 ~]# cat /etc/ceph/iscsi-gateway.cfg
[config]
# Name of the Ceph storage cluster. A suitable Ceph
configuration file allowing
# access to the Ceph storage cluster from the gateway
node is required, if not
# colocated on an OSD node.
cluster_name = ceph
# Place a copy of the ceph cluster's admin keyring in
the gateway's /etc/ceph
# drectory and reference the filename here
gateway_keyring = ceph.client.admin.keyring
# API settings.
# The API supports a number of options that allow you to
tailor it to your
# local environment. If you want to run the API under
https, you will need to
# create cert/key files that are compatible for each
iSCSI gateway node, that is
# not locked to a specific node. SSL cert and key files
*must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed
in the '/etc/ceph/' directory
# on *each* gateway node. With the SSL files in place,
you can use 'api_secure = true'
# to switch to https mode.
# To support the API, the bare minimum settings are:
api_secure = false
# Additional API configuration options are as follows,
defaults shown.
api_user = admin
api_password = admin
api_port = 5001
# API IP
trusted_ip_list = 172.16.200.251,172.16.200.252
--
Best Regards,
Xiubo Li (李秀波)
Email:xiubli@redhat.com/xiubli@ibm.com
Slack: @Xiubo Li
440
days inactive
443
days old
8 comments
2 participants
participants (2)
-
luckydog xf -
Xiubo Li