Styjun

Basic power tool set for Home repair and simple projects

Fibonacci sequence and other metallic sequences emerged in the form of fractions

Digital signature that is only verifiable by one specific person

What kind of chart is this?

How to make all magic-casting innate, but still rare?

Is it a bad idea to have a pen name with only an initial for a surname?

How do credit card companies know what type of business I'm paying for?

Should I email my professor to clear up a (possibly very irrelevant) awkward misunderstanding?

Having some issue with notation in a Hilbert space

Expand command in an argument before the main command

How do I become a better writer when I hate reading?

In a Fish that is not a Fish

Is a sequel allowed to start before the end of the first book?

How can the US president give an order to a civilian?

How did Frodo know where the Bree village was?

Do details of my undergraduate title matter?

You may find me... puzzling

I have found ports on my Samsung smart tv running a display service. What can I do with it?

How did space travel spread throughout the Star Wars galaxy?

How to prevent cables getting intertwined

Can I define two primary keys in a database table & why?

How can caller ID be faked?

Harmonic Series Phase Difference?

Probability Dilemma

How can I submit another task_list after one task_list finished in asyncio module?

Asyncio.gather vs asyncio.waitHow can I represent an 'Enum' in Python?How to import a module given the full path?How can I safely create a nested directory?How can I remove a trailing newline in Python?How to return multiple values from a function?How can I make a time delay in Python?How can you profile a Python script?How can I get a list of locally installed Python modules?How can I count the occurrences of a list item?“Large data” work flows using pandas

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;

1

I'm new to asyncio module， pls pardon me for my bad trial:

In my codes, I try to submit 100 tasks at first, then submit next 100 tasks after fore-100 tasks finished, then another next 100 tasks.

What should I do to make it work?

# len(ad_accounts) = 1000 for example 
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in ad_accounts[:100]
 ]
#just submit 100 tasks here once
new_loop.run_until_complete(asyncio.wait(task_list))
# then can I continuely submit next 100 tasks?

python python-asyncio

share|improve this question

edited Mar 25 at 5:46

jia Jimmy

asked Mar 25 at 4:48

jia Jimmyjia Jimmy

525316

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You are doing correctly, but should use get_event_loop instead of new and set. What's the problem you have?
  
  – MatrixTai
  Mar 25 at 5:46
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @MatrixTai , thanks for replying, I just want to run part of 1000 tasks (100 for example) at once , and run next 100 tasks after fore 100 tasks finished..
  
  – jia Jimmy
  Mar 25 at 6:22
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @MatrixTai Is this works ? loop.run_until_complete(asyncio.wait(task_list_100)) loop.run_until_complete(asyncio.wait(task_list_200)) loop.run_until_complete(asyncio.wait(task_list_300))...
  
  – jia Jimmy
  Mar 25 at 6:23
  

  
  
  
  

add a comment | 

1

I'm new to asyncio module， pls pardon me for my bad trial:

In my codes, I try to submit 100 tasks at first, then submit next 100 tasks after fore-100 tasks finished, then another next 100 tasks.

What should I do to make it work?

# len(ad_accounts) = 1000 for example 
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in ad_accounts[:100]
 ]
#just submit 100 tasks here once
new_loop.run_until_complete(asyncio.wait(task_list))
# then can I continuely submit next 100 tasks?

python python-asyncio

share|improve this question

edited Mar 25 at 5:46

jia Jimmy

asked Mar 25 at 4:48

jia Jimmyjia Jimmy

525316

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  You are doing correctly, but should use get_event_loop instead of new and set. What's the problem you have?
  
  – MatrixTai
  Mar 25 at 5:46
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @MatrixTai , thanks for replying, I just want to run part of 1000 tasks (100 for example) at once , and run next 100 tasks after fore 100 tasks finished..
  
  – jia Jimmy
  Mar 25 at 6:22
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @MatrixTai Is this works ? loop.run_until_complete(asyncio.wait(task_list_100)) loop.run_until_complete(asyncio.wait(task_list_200)) loop.run_until_complete(asyncio.wait(task_list_300))...
  
  – jia Jimmy
  Mar 25 at 6:23
  

  
  
  
  

add a comment | 

I'm new to asyncio module， pls pardon me for my bad trial:

In my codes, I try to submit 100 tasks at first, then submit next 100 tasks after fore-100 tasks finished, then another next 100 tasks.

What should I do to make it work?

# len(ad_accounts) = 1000 for example 
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in ad_accounts[:100]
 ]
#just submit 100 tasks here once
new_loop.run_until_complete(asyncio.wait(task_list))
# then can I continuely submit next 100 tasks?

python python-asyncio

share|improve this question

edited Mar 25 at 5:46

jia Jimmy

asked Mar 25 at 4:48

jia Jimmyjia Jimmy

525316

# len(ad_accounts) = 1000 for example 
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in ad_accounts[:100]
 ]
#just submit 100 tasks here once
new_loop.run_until_complete(asyncio.wait(task_list))
# then can I continuely submit next 100 tasks?

share|improve this question

edited Mar 25 at 5:46

jia Jimmy

asked Mar 25 at 4:48

jia Jimmyjia Jimmy

525316

share|improve this question

edited Mar 25 at 5:46

jia Jimmy

asked Mar 25 at 4:48

jia Jimmyjia Jimmy

525316

asked Mar 25 at 4:48

jia Jimmyjia Jimmy

525316

asked Mar 25 at 4:48

jia Jimmyjia Jimmy

525316

1 Answer
1

active

oldest

votes

1

Few things to add,

1. get_event_loop will try to access any available event loop, if there is not, it will call new_event_loop with set_event_loop, [Doc].




2. Simply use asyncio.gather when you won't do any further actions individual task. Use gather provides you a way to stop whole task group as well, see this SO answer.




3. You may notice in the following, () generator is used instead of [] list comprehension, as you don't need the list actually be saved in memory, and you don't need it until iteration. Change back to [] bracket if it is not the case.

Whole working example will be:

# len(ad_accounts) = 1000 for example

chunk_size = 100
batched_tasks = (ad_accounts[i:i + chunk_size] for i in range(0, len(ad_accounts), chunk_size))
_loop = asyncio.get_event_loop()

for task_group in batched_tasks:
 task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in task_group
 ]
 #just submit 100 tasks here once
 _loop.run_until_complete(asyncio.gather(*task_list))
 # Or _loop.run_until_complete(asyncio.wait(task_list))

share|improve this answer

answered Mar 25 at 7:52

MatrixTaiMatrixTai

2,1161723

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  thanks for your patience, benefit a lot from your words. :)
  
  – jia Jimmy
  Mar 25 at 9:10
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55331403%2fhow-can-i-submit-another-task-list-after-one-task-list-finished-in-asyncio-modul%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

1

Few things to add,

1. get_event_loop will try to access any available event loop, if there is not, it will call new_event_loop with set_event_loop, [Doc].




2. Simply use asyncio.gather when you won't do any further actions individual task. Use gather provides you a way to stop whole task group as well, see this SO answer.




3. You may notice in the following, () generator is used instead of [] list comprehension, as you don't need the list actually be saved in memory, and you don't need it until iteration. Change back to [] bracket if it is not the case.

Whole working example will be:

# len(ad_accounts) = 1000 for example

chunk_size = 100
batched_tasks = (ad_accounts[i:i + chunk_size] for i in range(0, len(ad_accounts), chunk_size))
_loop = asyncio.get_event_loop()

for task_group in batched_tasks:
 task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in task_group
 ]
 #just submit 100 tasks here once
 _loop.run_until_complete(asyncio.gather(*task_list))
 # Or _loop.run_until_complete(asyncio.wait(task_list))

share|improve this answer

answered Mar 25 at 7:52

MatrixTaiMatrixTai

2,1161723

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  thanks for your patience, benefit a lot from your words. :)
  
  – jia Jimmy
  Mar 25 at 9:10
  

  
  
  
  

add a comment | 

1

Few things to add,

1. get_event_loop will try to access any available event loop, if there is not, it will call new_event_loop with set_event_loop, [Doc].




2. Simply use asyncio.gather when you won't do any further actions individual task. Use gather provides you a way to stop whole task group as well, see this SO answer.




3. You may notice in the following, () generator is used instead of [] list comprehension, as you don't need the list actually be saved in memory, and you don't need it until iteration. Change back to [] bracket if it is not the case.

Whole working example will be:

# len(ad_accounts) = 1000 for example

chunk_size = 100
batched_tasks = (ad_accounts[i:i + chunk_size] for i in range(0, len(ad_accounts), chunk_size))
_loop = asyncio.get_event_loop()

for task_group in batched_tasks:
 task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in task_group
 ]
 #just submit 100 tasks here once
 _loop.run_until_complete(asyncio.gather(*task_list))
 # Or _loop.run_until_complete(asyncio.wait(task_list))

share|improve this answer

answered Mar 25 at 7:52

MatrixTaiMatrixTai

2,1161723

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  thanks for your patience, benefit a lot from your words. :)
  
  – jia Jimmy
  Mar 25 at 9:10
  

  
  
  
  

add a comment | 

Few things to add,

1. get_event_loop will try to access any available event loop, if there is not, it will call new_event_loop with set_event_loop, [Doc].




2. Simply use asyncio.gather when you won't do any further actions individual task. Use gather provides you a way to stop whole task group as well, see this SO answer.




3. You may notice in the following, () generator is used instead of [] list comprehension, as you don't need the list actually be saved in memory, and you don't need it until iteration. Change back to [] bracket if it is not the case.

Whole working example will be:

# len(ad_accounts) = 1000 for example

chunk_size = 100
batched_tasks = (ad_accounts[i:i + chunk_size] for i in range(0, len(ad_accounts), chunk_size))
_loop = asyncio.get_event_loop()

for task_group in batched_tasks:
 task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in task_group
 ]
 #just submit 100 tasks here once
 _loop.run_until_complete(asyncio.gather(*task_list))
 # Or _loop.run_until_complete(asyncio.wait(task_list))

share|improve this answer

answered Mar 25 at 7:52

MatrixTaiMatrixTai

2,1161723

# len(ad_accounts) = 1000 for example

chunk_size = 100
batched_tasks = (ad_accounts[i:i + chunk_size] for i in range(0, len(ad_accounts), chunk_size))
_loop = asyncio.get_event_loop()

for task_group in batched_tasks:
 task_list = [
 asyncio.ensure_future(_handle_account(account)) for ad_account in task_group
 ]
 #just submit 100 tasks here once
 _loop.run_until_complete(asyncio.gather(*task_list))
 # Or _loop.run_until_complete(asyncio.wait(task_list))

share|improve this answer

answered Mar 25 at 7:52

MatrixTaiMatrixTai

2,1161723

answered Mar 25 at 7:52

MatrixTaiMatrixTai

2,1161723

answered Mar 25 at 7:52

MatrixTaiMatrixTai

2,1161723