Re: [argobots-discuss] externally allocated stacks and unnamed (detached) threads
Phil, Shintaro, Just in case, we have implemented it in DAOS, and it works just fine ! You can find the whole code for the feature at https://github.com/daos-stack/daos/pull/5111 . Cheers, Bruno. From: Shintaro Iwasaki via discuss <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday 18 November 2021 at 02:29 To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Cc: Shintaro Iwasaki <[email protected]<mailto:[email protected]>> Subject: Re: [argobots-discuss] externally allocated stacks and unnamed (detached) threads Hi Phil, Thanks for your question! Calling a custom stack destructor for an unnamed ULT is possible within the current Argobots infrastructure. This is related to this issue: https://github.com/pmodels/argobots/issues/16. For the detailed implementation, please find the complete code here (collapsed): https://github.com/pmodels/argobots/issues/274#issuecomment-785221242. Though you have to call init_stack_key() and finalize_stack_key() to use this, it should be simple enough: you just basically change allocate_stack() and free_stack() for your workload. I believe this is the quick path to implement your idea. But maybe a more intuitive stack-free-hook interface would be preferable (this request is not the first time). Please let me know if you need such a new interface (especially if your idea works well). Thanks, Shintaro On Wed, Nov 17, 2021 at 2:15 PM Phil Carns via discuss <[email protected]<mailto:[email protected]>> wrote: Hi all, We've been thinking about some memory optimizations for which it might be easier if our own code provided the stack memory for newly created ULTs so that we have full control over the allocation strategy. This seems pretty straightforward with the ABT_thread_attr_set_stack() function: https://www.argobots.org/doxygen/latest/d2/df6/group__ULT__ATTR.html#gac8f39... However, our use case often creates unnamed (detached) threads by setting the ABT_thread_create() newthread argument to NULL. I can elaborate if needed, but the short story is that this is a really natural fit for RPC handlers in our model: https://www.argobots.org/doxygen/latest/d0/d6d/group__ULT.html#ga73bd1f04ce5... Is it possible to do both (provide an externally managed stack pointer, and use unnamed ULTs)? From the documentation it looks like the answer is probably no, because the caller who created the stack allocation doesn't explicitly free the ULT, and thus doesn't have a way to free the stack after the ULT is complete. I wanted to double check though before I start thinking about alternatives, though. thanks! -Phil _______________________________________________ discuss mailing list [email protected]<mailto:[email protected]> https://lists.argobots.org/mailman/listinfo/discuss --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 4,572,000 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Fantastic, thank you for the pointers, Shintaro and Bruno! We'll follow this model and let you know if we have feedback. thanks, -Phil On 11/18/21 9:37 AM, Faccini, Bruno via discuss wrote:
Phil, Shintaro,
Just in case, we have implemented it in DAOS, and it works just fine !
You can find the whole code for the feature at https://github.com/daos-stack/daos/pull/5111 .
Cheers,
Bruno.
**
**
*From:*Shintaro Iwasaki via discuss <[email protected] <mailto:[email protected]>> *Reply-To:*"[email protected] <mailto:[email protected]>" <[email protected] <mailto:[email protected]>> *Date:*Thursday 18 November 2021 at 02:29 *To:*"[email protected] <mailto:[email protected]>" <[email protected] <mailto:[email protected]>> *Cc:*Shintaro Iwasaki <[email protected] <mailto:[email protected]>> *Subject:*Re: [argobots-discuss] externally allocated stacks and unnamed (detached) threads
Hi Phil,
Thanks for your question!
Calling a custom stack destructor for an unnamed ULT is possible within the current Argobots infrastructure. This is related to this issue:https://github.com/pmodels/argobots/issues/16 <https://github.com/pmodels/argobots/issues/16>.
For the detailed implementation, please find the complete code here (collapsed):https://github.com/pmodels/argobots/issues/274#issuecomment-785221242 <https://github.com/pmodels/argobots/issues/274#issuecomment-785221242>. Though you have to call init_stack_key() and finalize_stack_key() to use this, it should be simple enough: you just basically change allocate_stack() and free_stack() for your workload.
I believe this is the quick path to implement your idea. But maybe a more intuitive stack-free-hook interface would be preferable (this request is not the first time). Please let me know if you need such a new interface (especially if your idea works well).
Thanks,
Shintaro
On Wed, Nov 17, 2021 at 2:15 PM Phil Carns via discuss <[email protected] <mailto:[email protected]>> wrote:
Hi all,
We've been thinking about some memory optimizations for which it might be easier if our own code provided the stack memory for newly created ULTs so that we have full control over the allocation strategy. This seems pretty straightforward with the ABT_thread_attr_set_stack() function:
https://www.argobots.org/doxygen/latest/d2/df6/group__ULT__ATTR.html#gac8f39... <https://www.argobots.org/doxygen/latest/d2/df6/group__ULT__ATTR.html#gac8f39be20d677898c45293c5ce31eea0>
However, our use case often creates unnamed (detached) threads by setting the ABT_thread_create() newthread argument to NULL. I can elaborate if needed, but the short story is that this is a really natural fit for RPC handlers in our model:
https://www.argobots.org/doxygen/latest/d0/d6d/group__ULT.html#ga73bd1f04ce5... <https://www.argobots.org/doxygen/latest/d0/d6d/group__ULT.html#ga73bd1f04ce57def336cb20cdec9b14bd>
Is it possible to do both (provide an externally managed stack pointer, and use unnamed ULTs)? From the documentation it looks like the answer is probably no, because the caller who created the stack allocation doesn't explicitly free the ULT, and thus doesn't have a way to free the stack after the ULT is complete.
I wanted to double check though before I start thinking about alternatives, though.
thanks!
-Phil
_______________________________________________ discuss mailing list [email protected] <mailto:[email protected]> https://lists.argobots.org/mailman/listinfo/discuss <https://lists.argobots.org/mailman/listinfo/discuss>
--------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 4,572,000 Euros
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________ discuss mailing list [email protected] https://lists.argobots.org/mailman/listinfo/discuss
participants (2)
-
Faccini, Bruno -
Phil Carns