隐藏

c# – 如何限制并发异步I/O操作的数量?

发布:2021/3/6 9:10:20作者:管理员 来源:本站 浏览次数:959

// let's say there is a list of 1000+ URLs
string[] urls = { "http://google.com", "http://yahoo.com", ... };

// now let's send HTTP requests to each of these URLs in parallel
urls.AsParallel().ForAll(async (url) => {
    var client = new HttpClient();
    var html = await client.GetStringAsync(url);
});

这里是问题,它启动1000同时Web请求。是否有一个简单的方法来限制这些异步http请求的并发量?所以在任何给定的时间下载不超过20个网页。如何以最有效的方式做到这一点?

你绝对可以在最新版本的async for .NET中使用.NET 4.5 Beta来做到这一点。从’usr’的前一篇文章指向一个由Stephen Toub编写的好文章,但是较少公布的消息是,异步信号量实际上使其成为.NET 4.5的测试版

如果你看看我们所爱的SemaphoreSlim类(你应该使用它,因为它比原来的Semaphore更高性能),它现在拥有WaitAsync(...)系列的重载,所有的期望参数 – 超时间隔,取消令牌,所有的平常调度好友:)

Stephen也写了一个更新的博客文章关于新的.NET 4.5好看的测试见What’s New for Parallelism in .NET 4.5 Beta

最后,这里有一些关于如何使用SemaphoreSlim进行异步方法限制的示例代码:

public async Task MyOuterMethod()
{
    // let's say there is a list of 1000+ URLs
    var urls = { "http://google.com", "http://yahoo.com", ... };

    // now let's send HTTP requests to each of these URLs in parallel
    var allTasks = new List<Task>();
    var throttler = new SemaphoreSlim(initialCount: 20);
    foreach (var url in urls)
    {
        // do an async wait until we can schedule again
        await throttler.WaitAsync();

        // using Task.Run(...) to run the lambda in its own parallel
        // flow on the threadpool
        allTasks.Add(
            Task.Run(async () =>
            {
                try
                {
                    var client = new HttpClient();
                    var html = await client.GetStringAsync(url);
                }
                finally
                {
                    throttler.Release();
                }
            }));
    }

    // won't get here until all urls have been put into tasks
    await Task.WhenAll(allTasks);

    // won't get here until all tasks have completed in some way
    // (either success or exception)
}

 

最后,但可能值得一提的是使用基于TPL的调度的解决方案。您可以在TPL上创建尚未启动的委托绑定任务,并允许自定义任务调度程序限制并发。事实上,在这里有一个MSDN样本:

参见TaskScheduler