由于抓取可能是一项资源繁重且运行时间长的任务,因此您应该将其委托给排队的作业。您可以在此处阅读有关队列和作业的信息:https://laravel.com/docs/master/queues
我建议将用户的抓取请求保存为模型条目。这样您就可以跟踪状态(正在运行、出错、已完成等),将其显示在任务列表中,将系统负载归因于用户,等等。类似于ScrapeTask(id、user_id、url、status、时间戳以及您需要的任何其他字段)。
设置好模型和队列后,您将创建一个作业类。这可以通过运行php artisan make:job ScrapeWebsite 来完成,也可以手动创建。
app/Jobs/ScrapeWebsite.php
<?php
namespace App\Jobs;
use App\ScrapeTask;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
class ScrapeWebsite implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/** @var ScrapeTask */
protected $scrapeTask;
/**
* Create a new job instance.
*
* @param ScrapeTask $scrapeTask
*/
public function __construct(ScrapeTask $scrapeTask)
{
$this->scrapeTask = $scrapeTask;
// also inject any necessary third party libraries, etc.
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
// scrape the website at $this->scrapeTask->url
}
}
你的控制器方法看起来像这样:
app/Http/Controllers/ScraperController.php
<?php
namespace App\Http\Controllers;
use Auth;
use App\ScrapeTask;
use App\Jobs\ScrapeWebsite;
use Illuminate\Http\Request;
class ScraperController extends Controller
{
public function postScrape(Request $request)
{
// perform input validation, etc.
// create a new scrape task
$scrapeTask = ScrapeTask::create([
'user_id' => Auth::id(),
'url' => $request->input('url'),
]);
// dispatch the ScrapeWebsite job to the queue
dispatch(new ScrapeWebsite($scrapeTask));
// redirect to a scrape status monitoring page (or do whatever...)
return redirect()->route('scrape.monitor', $scrapeTask->id);
}
}
我强烈建议您设置失败的工作跟踪 (https://laravel.com/docs/master/queues#dealing-with-failed-jobs),以确保您知道什么时候出现问题。