hi guys,
i just came across boost::outcome[0]. it reminded me the discussion we
had back in Barcelona regarding to the error handling in crimson.
well, strictly speaking, it's not limited to errors. it covers the
non-error handling as well.
the question is: shall we start prototyping the crimson variant of
outcome<> now? if yes, probably we can leverage boost::outcome<>?
a little bit background:
seastar uses exception for propagating the error. but it incurs
runtime overhead. because, to throw an exception, the libstdc++
runtime needs to acquire a global lock.
well, some of us might want to argue, why not just return a
future<Result, Error>? let me use an example here, imagine we are
handling a write request in OSD. we might need to go through following
steps:
1. perform some sanity tests, for instance, to see if the OSD is ready
for handling the write request
2. try to read the object info of the object from local storage to see
if it already exists
3. write to the object to the local storage, and send write requests
to replica OSDs (assuming it's in a replicated pool), wait for the
completions of these write ops.
4. update the statistics
5. reply to the client
and it's intuitive to structure these steps using chained continuation like
do_with(std::move(request), [this](auto request) {
return perform_tests(request->object_id).then([request, this] {
return read_object_info(request->object_id);
}).then([request, this](optional<object_info> object_info) {
return when_all(
write_local(request->object_id, request->offset, request->data),
parallel_for_each(replica_osds, [request](auto replica_osd) {
return replica_osd->write_remote(request->object_id,
request->offset, request->data)
}));
}).then([write_size=request.data.size(), this] {
update_statistics(write_size);
return reply_to(reply_t::success, request);
});
}).handle_exception([](auto exception) {
return reply_to(reply_t::failure, exception.error_code, request);
});
in which, if any test fails in step#1, we either need to wait until
the OSD is ready, or just need to bail out, and skip the following
steps. the "handle_exception()" clause is used to handle the "bail
out" case, where we cannot do anything to serve the request. for
instance, the request is invalid.
we want to differentiate two types of errors. one of them are actually
exceptions which does not happen often in real world, and we don't
need/want to optimize for this case. but the other case could be
normal. for instance, it's fairly normal that an object does not exist
yet, when we are trying to write to it. and we do want to be
performant when handling these "errors" in this category, and also, we
want to do this in a convenient way just like handling exceptions.
because, we need an efficient way to convey the message to caller that
"please skip the following continuations, and i would go to this
handling route instead". if my memory serves me correctly, we think
that we need to create a wrapper around seastar::future<> to allow the
caller to do something like
// a helper to run func or skip it
template<typename Func>
auto ignore_on_error(Func&& f) {
return [f=std::move(f)](auto&& t) {
return t.is_value() ? f(t.value()) ? t;
}
}
return read_object_info(oid).then(
return ignore_on_error([](object_info& oi) {
return handle_write_with_object_info(std::move(oi));
}).then([](auto t) {
return handle_write_without_object_info();
});
);
in the example above, i assume we will do something very different
depending on if the object's existence.
cheers,
---
[0] https://www.boost.org/doc/libs/1_70_0/libs/outcome/doc/html/index.html
--
Regards
Kefu Chai
The next Ceph Developer Monthly falls on this Wednesday, July 3. Since
this is adjacent to a US holidy it's likely many people won't make it.
More importantly, we failed to send out an agenda last week.
Let's delay this until next week, Jul 10 9PM ET (Jul 11 0100 UTC).
Thanks!
sage
hi Mark,
i am working on using cbt for testing crimson. as you might known,
crimson-osd is currently using a variant of memstore as its object
store backend. so it'd be very easy for crimson-osd to run out of
memory, as the default run "time" of cbt radosbench is 300 seconds.
currently, each radosbench run is composed of 3 steps:
1. prefill // optional, enabled if "prefill_time" or "prefill_objects" is set
2. write.
3. read // optional, enabled if "write_only" is not test
the pain point is that the run times for write and read step are
specified using the same setting -- "time".
so, i am wondering if it's okay to add an option named "read_only" to
skip the "write" step to let the prefill to prepare the testbench for
the read test. so we can specify the time for prefilling and the time
for read separately?
as an alternative, we could have an option for "write_time", which
defaults to "time" if not specified, but if it takes precedence over
"time" if specified. and it's "0", the "write" step will be skipped.
what do you think?
--
Regards
Kefu Chai
Hi Ernesto Puerta,
Thanks for your great helping hand, and rapid response.
It is my honor to join the biweekly meeting.
Is OFTC be the chanel for ircs://irc.oftc.net:6697? Ok, I will cc
dev(a)ceph.io later.
I have three problem below when I research the code of orchestrator and
backend:
1.Can I copy the nautilus's module file(ansible, orchestrator) inside
mimic's mgr file?Can it work?
2.I found that the function of create_osds(self, drive_group, all_hosts) in
ansible module can deal my problem. But I don't know the way to use it.
Should I add the function to call inside dashboard module.py(I don't know
whether the main function for dashabord is here or not)? Then using this
function to get(or post) the query from js in frontend? If I get the query
then using the function from ansible module.
3.I try to use ./run-tox.sh for backend but show the Permission denied
error.
Please give me any suggestion and comment, all of your advise is very
valuable.
Thank you very much
Best regards
Hausiowen Nayu
Ernesto Puerta <epuertat(a)redhat.com> 於 2019年7月1日 週一 下午4:29寫道：
> Hi Hausiowen,
>
> We saw you connecting and dropping, so we assumed you had issues.
> Bluejeans some times requires some tuning before it works properly. You can
> join our biweekly meeting, you can join OFTC #ceph-dashboard IRC channel as
> well (this is the preferred method for quick ask&answers). Additionally, we
> could go back to dev(a)ceph.io. I think other people might benefit from
> this discussion, if it's held publicly.
>
> On your specific questions: I think some support is available for
> orchestrators in Mimic, but probably not the best. The idea is that you
> choose your preferred orchestrator tool (kubernets/rook, Ansible, Deepsea,
> or the in-progress SSH orchestrator) and then you let one of these interact
> with Ceph by means of a generic command line interface.
>
> Kind regards,
>
> Ernesto Puerta
>
> He / Him / His
>
> Senior Software Engineer, Ceph
>
> Red Hat <https://www.redhat.com/>
> <https://www.redhat.com/>
>
>
> On Mon, Jul 1, 2019 at 4:58 AM seribe pasada <clear60enator(a)gmail.com>
> wrote:
>
>> Hi Ernesto Puerta,
>>
>> Thanks for your great helping hand, and rapid response.
>> And thank you so much for the invitation, I really appreciate it and it
>> means a great deal. But I can't listen any voice when the meeting. Maybe I
>> should find the way to deal this problem. May I join you next biweekly
>> meeting?
>>
>> I have research the code of orchestrator_modules, however, the module
>> seem to support the nautilus only. So I don't know if it support mimic or
>> not.(or using way to add module to mimic?) In addition, the feature of
>> adding OSD in orchestrator seem to be necessary with deepsea and ansible.
>> Should I install the deepsea(only support opensuse) and ansible
>> additionally?
>>
>>
>> Thank you very much
>>
>> Best regards
>> Hausiowen Nayu
>>
>> Ernesto Puerta <epuertat(a)redhat.com> 於 2019年6月28日 週五 下午8:05寫道：
>>
>>> Hi Hausiowen,
>>>
>>> Aha, so you want to add to the dashboard the capacity for creating new
>>> OSDs, right?
>>>
>>> I don't know if you are familiar to this, but there's an ongoing effort
>>> to implement this kind of functionality via a neutral/abstract orchestrator
>>> interface (http://docs.ceph.com/docs/master/mgr/orchestrator_modules/
>>> and http://docs.ceph.com/docs/master/mgr/orchestrator_cli/) and
>>> different implementations with 3rd party orchestrators (Rook, Ansible,
>>> DeepSea, SSH Orchestrator).
>>>
>>> However, I don't think there's any effort to bring ceph-deploy into that
>>> pack. I'm CCing Dashboard (Lenz) and Orchestrator (Sebastian) leads. You're
>>> welcome to join this existing effort. Today at 15.00 CEST we have our
>>> Dashboard biweekly sync meeting (https://bluejeans.com/633233233), so
>>> feel free to drop by, introduce yourself and bring in the discussion. We'll
>>> be glad to have you there.
>>>
>>> Kind regards,
>>>
>>> Ernesto Puerta
>>>
>>> He / Him / His
>>>
>>> Senior Software Engineer, Ceph
>>>
>>> Red Hat <https://www.redhat.com/>
>>> <https://www.redhat.com/>
>>>
>>>
>>> On Fri, Jun 28, 2019 at 12:36 PM seribe pasada <clear60enator(a)gmail.com>
>>> wrote:
>>>
>>>> Hi Ernesto Puerta,
>>>>
>>>> Thanks for your great helping hand, and rapid response.
>>>> I can add the link to the page successfully with my component and
>>>> service providing by myself now.
>>>> It's very appreciated you for your big hand, otherwise, I will spend
>>>> more time to solve the problem.
>>>>
>>>> I want to add button which can execute the shell command " ceph-deploy
>>>> osd create node11 --data /dev/sda ", so that I can touch button to
>>>> add osd.
>>>> So, I plan to add the python code in backend to execute the shell
>>>> command. Then, I add the button listening with java-scripts in component.
>>>> Finally, the python as backend get the request of java-scripts code as
>>>> frontend.
>>>> The command can be executed when I touch button.
>>>>
>>>> Is it the good idea for add osd using button?
>>>> If I want to add the pyhon code in backend, should I compile with
>>>> ./run-backend-api-tests.sh?
>>>>
>>>>
>>>> Thank you very much
>>>>
>>>> Best regards
>>>> Hausiowen Nayu
>>>>
>>>> Ernesto Puerta <epuertat(a)redhat.com> 於 2019年6月28日 週五 下午6:10寫道：
>>>>
>>>>> Controller? You mean back-end or front-end?
>>>>>
>>>>> You need to do these 2 things:
>>>>> - Add the link in the navigation component
>>>>> - Add the component and exposing it in an existing/new module
>>>>> - Add that component to the router (you may use the main router module)
>>>>>
>>>>> If you want to interact with some back-end (server-side python) code,
>>>>> you need to use a controller.
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> Ernesto Puerta
>>>>>
>>>>> He / Him / His
>>>>>
>>>>> Senior Software Engineer, Ceph
>>>>>
>>>>> Red Hat <https://www.redhat.com/>
>>>>> <https://www.redhat.com/>
>>>>>
>>>>>
>>>>> On Fri, Jun 28, 2019 at 5:26 AM seribe pasada <clear60enator(a)gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ernesto Puerta,
>>>>>>
>>>>>> Thanks for your great helping hand, and rapid response.
>>>>>> I have tried add the new button in /navigation.component.html and
>>>>>> service api.It's sucessfully.
>>>>>> But the page is 404 because I haven't add the page to controller. So
>>>>>> I guess the controller is in component?
>>>>>> Do you know how to add page in the router link?
>>>>>>
>>>>>>
>>>>>> Thank you very much
>>>>>>
>>>>>> Best regards
>>>>>> Hausiowen Nayu
>>>>>>
>>>>>> Ernesto Puerta <epuertat(a)redhat.com> 於 2019年6月27日 週四 下午5:53寫道：
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> To add a new link to the navigation, you just need to modify the
>>>>>>> following HTML file:
>>>>>>> src/pybing/mgr/dashboard/frontend/src/app/core/navigation/navigation/navigation.component.html
>>>>>>>
>>>>>>> If you have a look at that code, you'll quickly see how top-level,
>>>>>>> and second-level menu entries are added.
>>>>>>>
>>>>>>> Good luck with it!
>>>>>>>
>>>>>>> Ernesto Puerta
>>>>>>>
>>>>>>> He / Him / His
>>>>>>>
>>>>>>> Senior Software Engineer, Ceph
>>>>>>>
>>>>>>> Red Hat <https://www.redhat.com/>
>>>>>>> <https://www.redhat.com/>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 27, 2019 at 11:46 AM seribe pasada <
>>>>>>> clear60enator(a)gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Ernesto Puerta,
>>>>>>>>
>>>>>>>> Thanks for your great helping hand
>>>>>>>>
>>>>>>>> Why I want are three parts:
>>>>>>>> add the new page in first or secondary layer, which can show
>>>>>>>> in the origin dashboard
>>>>>>>> I want to add the link of other website to the page for
>>>>>>>> dashboard
>>>>>>>> I can add the button in the page.When I touch the button, the
>>>>>>>> x.sh under /root can be executed
>>>>>>>>
>>>>>>>> The code I modified is show as below,
>>>>>>>> The "src file" including all of code, the "modified file" only
>>>>>>>> for the code I modified
>>>>>>>> The code can be compiled but the page of addOSD is wrong(404).
>>>>>>>> Thank you very much
>>>>>>>>
>>>>>>>> Best regards
>>>>>>>> Hausiowen Nayu
>>>>>>>>
>>>>>>>> Ernesto Puerta <epuertat(a)redhat.com> 於 2019年6月26日 週三 下午6:37寫道：
>>>>>>>>
>>>>>>>>> Hi Hausiowen,
>>>>>>>>>
>>>>>>>>> Glad to have helped you!
>>>>>>>>>
>>>>>>>>> Could I know what are you trying to do? So I can better understand
>>>>>>>>> this.
>>>>>>>>>
>>>>>>>>> Just as a comment: Router Module is just about mapping
>>>>>>>>> (front-end/browser) URLs to Angular components. It has nothing to
>>>>>>>>> do
>>>>>>>>> with the main Navigation Drop-down Menu. That's actually hardcoded
>>>>>>>>> links (URLs) in the HTML template of Navigation Component
>>>>>>>>> (
>>>>>>>>> https://epuertat.github.io/components/NavigationComponent.html#template
>>>>>>>>> ).
>>>>>>>>>
>>>>>>>>> Kind regards,
>>>>>>>>>
>>>>>>>>> Ernesto Puerta
>>>>>>>>>
>>>>>>>>> He / Him / His
>>>>>>>>>
>>>>>>>>> Senior Software Engineer, Ceph
>>>>>>>>>
>>>>>>>>> Red Hat
>>>>>>>>>
>>>>>>>>> On Wed, Jun 26, 2019 at 10:00 AM seribe pasada <
>>>>>>>>> clear60enator(a)gmail.com> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi Ernesto Puerta,
>>>>>>>>> >
>>>>>>>>> > Nice to meet you
>>>>>>>>> > Thanks for your great helping hand
>>>>>>>>> > I can compile and build the code in mgr-dashboard then show it
>>>>>>>>> in web now
>>>>>>>>> > However, I got the problem when I try to add the new page in the
>>>>>>>>> button. I studied the structure of dashboard, but I still got error.
>>>>>>>>> > Maybe I have something mistake when I try to code.
>>>>>>>>> > I modify the code in src(source code) which including(the
>>>>>>>>> structure show as picture or link):
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > first layer router(the button:dashboard、cluster、pool...etc):
>>>>>>>>> /app/app.module.ts
>>>>>>>>> > all router: /app/app-routing.module.ts
>>>>>>>>> > secondar router(the button below first
>>>>>>>>> layer:cluster->hosts、monitor...etc): /app/ceph/cluster/cluster.module.ts
>>>>>>>>> > view: /app/ceph/cluster/hosts/hosts.component.ts
>>>>>>>>> > web foldere(temp+view): /app/ceph/cluster/hosts
>>>>>>>>> > service router: /app/shared/api/api.module.ts
>>>>>>>>> > router function: /app/shared/api/host.service.ts
>>>>>>>>> > router link:
>>>>>>>>> /app/core/navigation/navigation/navigation.component.html
>>>>>>>>> >
>>>>>>>>> > Can you tell me any I missing?
>>>>>>>>> >
>>>>>>>>> > Thank you very much
>>>>>>>>> >
>>>>>>>>> > Best regards
>>>>>>>>> > Hausiowen Nayu
>>>>>>>>> >
>>>>>>>>> > link:
>>>>>>>>> >
>>>>>>>>> https://drive.google.com/file/d/1Gyh_4K4iRMkEyxsHkQIgmCeeKBh1mRsw/view
>>>>>>>>> >
>>>>>>>>> > Ernesto Puerta <epuertat(a)redhat.com> 於 2019年6月24日 週一 下午11:15寫道：
>>>>>>>>> >>
>>>>>>>>> >> Hi Hausiowen,
>>>>>>>>> >>
>>>>>>>>> >> You can find (mostly) everything a developer may be interested
>>>>>>>>> in at
>>>>>>>>> >> Dashboard's HACKING.rst
>>>>>>>>> >> (
>>>>>>>>> https://github.com/ceph/ceph/blob/754e08017714543d71835c0deb3853603567f2b4/…
>>>>>>>>> ,
>>>>>>>>> >> I'm sending you that commit because Github is not rendering
>>>>>>>>> that doc
>>>>>>>>> >> in the master HEAD commit).
>>>>>>>>> >>
>>>>>>>>> >> You can delete all front-end code, and it will still work
>>>>>>>>> because
>>>>>>>>> >> Angular/Typescript code is 'transpiled' into Javascript code
>>>>>>>>> >> (frontend/dist subdir), so you don't directly run the 'code'
>>>>>>>>> version
>>>>>>>>> >> of the dashboard.
>>>>>>>>> >>
>>>>>>>>> >> So, answering your specific question, you may either:
>>>>>>>>> >> - Run from build/ subdir: "make
>>>>>>>>> mgr-dashboard-frontend-build", and
>>>>>>>>> >> then restart dashboard to reload new front-end ("ceph mgr module
>>>>>>>>> >> disable dashboard && ceph mgr module enable dashboard"),
>>>>>>>>> >> - Use the development mode front-end Angular provides, which
>>>>>>>>> >> performs incremental auto-builds every time you save a change:
>>>>>>>>> rename
>>>>>>>>> >> proxy.conf.json.sample to proxy.conf.json and add the backend
>>>>>>>>> port and
>>>>>>>>> >> HTTP/S proto and run "npm run start" (this will run a new
>>>>>>>>> front-end
>>>>>>>>> >> server in a different port than the ceph-mgr one).
>>>>>>>>> >>
>>>>>>>>> >> Let us know if that was of help to you!
>>>>>>>>> >>
>>>>>>>>> >> Kind Regards,
>>>>>>>>> >> Ernesto
>>>>>>>>> >>
>>>>>>>>> >> Ernesto Puerta
>>>>>>>>> >>
>>>>>>>>> >> He / Him / His
>>>>>>>>> >>
>>>>>>>>> >> Senior Software Engineer, Ceph
>>>>>>>>> >>
>>>>>>>>> >> Red Hat
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >>
>>>>>>>>> >> On Mon, Jun 24, 2019 at 4:37 PM seribe pasada <
>>>>>>>>> clear60enator(a)gmail.com> wrote:
>>>>>>>>> >> >
>>>>>>>>> >> > Hi dev,
>>>>>>>>> >> > Nice to meet you
>>>>>>>>> >> > I want to modify the code from ceph dashboard
>>>>>>>>> >> > I try to modify the code from
>>>>>>>>> /usr/lib/ceph/mgr/dashboard/frontend, but the web do noting even I delete
>>>>>>>>> all frontend folder.
>>>>>>>>> >> > I guess the frontend should built to the /frontend/dist
>>>>>>>>> folder by angular or nodejs; however, both of the methods is failed.
>>>>>>>>> >> > Can you tell me how to build or run the frontend code I
>>>>>>>>> modify to the web?
>>>>>>>>> >> >
>>>>>>>>> >> > Thank you very much
>>>>>>>>> >> >
>>>>>>>>> >> > Best regards
>>>>>>>>> >> > Hausiowen Nayu
>>>>>>>>> >> > _______________________________________________
>>>>>>>>> >> > Dev mailing list -- dev(a)ceph.io
>>>>>>>>> >> > To unsubscribe send an email to dev-leave(a)ceph.io
>>>>>>>>>
>>>>>>>>
