【发布时间】:2015-01-12 08:42:30
【问题描述】:
过去几天我一直在努力解决一个问题。 我有一个网站可以抓取新闻,而且效果很好。 然而,最近,我的analyzer_script 遇到了一个问题,因为它似乎超过了我的网络主机设置的时间限制。显然 max_execution 时间大约为 1 分钟,而我的脚本花费的时间比这更长。而且我无法在 php.ini 脚本中调整它,因为我将我的网站托管在公共服务器上。 我能做些什么?我需要重写我的脚本吗?
感谢您的帮助! 我的脚本如下:
<?php
$array = array();
$sub_array = array();
$analyzer_ids = array();
$res5 = mysqli_query($con,"SELECT id,status FROM statuz ORDER BY id DESC LIMIT 1");
$row5 = mysqli_fetch_array($res5);
$status = $row5['status'];
$status_id = $row5['id'];
if($status == 2) {
$res1 = mysqli_query($con,"SELECT tag, id FROM tags");
while($row1 = mysqli_fetch_array($res1)) {
$tag = $row1['tag'];
$id = $row1['id'];
$res2 = mysqli_query($con,"SELECT sub_tag FROM sub_tags WHERE tag_id = '$id'");
while($row2 = mysqli_fetch_array($res2)) {
$sub_tag = $row2['sub_tag'];
$sub_tag = strtolower($sub_tag);
$sub_array[] = $sub_tag;
}
$array[] = array('tag_id' => $id, 'tag' => $tag, 'sub_tag' => $sub_array);
$sub_array = array();
}
mysqli_query($con,"INSERT INTO analyzer_queue (crawler_id, status)
(SELECT id,0 FROM crawlers)");
$initial_res = mysqli_query($con,"SELECT crawler_id,id FROM analyzer_queue WHERE status = '0'");
while($initial_row = mysqli_fetch_array($initial_res)) {
$analyzer_id = $initial_row['id'];
$start_crawler_id = $initial_row['crawler_id'];
mysqli_query($con,"UPDATE analyzer_queue SET status = '1' WHERE crawler_id = '$start_crawler_id' ORDER BY id DESC LIMIT 1");
$analyzer_ids[] = $analyzer_id;
$res = mysqli_query($con,"SELECT cr.title, cr.content, cr.id
FROM crawler_results cr
INNER JOIN crawlers c
ON c.newspaper_id = cr.newspaper_id
WHERE c.id = '$start_crawler_id'
AND status = '3'
LIMIT 10");
while($row = mysqli_fetch_array($res)) {
$article_id = $row['id'];
$title = $row['title'];
$content = $row['content'];
$content = strip_tags($content);
$content = strtolower($content);
$title = strtolower($title);
$count = array();
foreach ($array as $tag) {
$regex = '/(?:\b' . preg_quote($tag['tag'], '/');
foreach ($tag['sub_tag'] as $sub) {
$regex .= '\b)|(?:\b' . preg_quote($sub, '/');
}
$regex .= '\b)/i';
$count_content = preg_match_all($regex, $content, $count_content);
$count_title = preg_match_all($regex, $title, $count_title);
$count_total[$tag['tag']] = $count_content + $count_title;
$total_count = $count_total[$tag['tag']];
$tag_name = $tag['tag'];
$res5 = mysqli_query($con,"SELECT id FROM tags WHERE tag = '$tag_name'");
$row5 = mysqli_fetch_array($res5);
$tag_id = $row5['id'];
if($total_count != 0) {
mysqli_query($con,"INSERT INTO article_tags (article_id,tag_id,count_tags) VALUES('$article_id','$tag_id','$total_count')");
}
echo$count_total[$tag['tag']];
echo"<br /><br />";
}
echo"<pre>";
print_r($count_total);
echo"</pre>";
mysqli_query($con,"UPDATE crawler_results SET status = '2', analyzer_id = '$analyzer_id' WHERE id = '$article_id'");
}
mysqli_query($con,"UPDATE analyzer_queue SET status = '2' WHERE crawler_id = '$start_crawler_id' ORDER BY id DESC LIMIT 1");
}
mysqli_query($con,"UPDATE crawler_results SET status = '4' WHERE analyzer_id NOT IN (".implode(',',$analyzer_ids).")");
mysqli_query($con,"UPDATE statuz SET status = '3' WHERE id = '$status_id'");
print_r($analyzer_ids);
} else {
echo"Not ready yet";
}
?>
【问题讨论】:
-
使用
mysqli时,您应该使用参数化查询和bind_param将用户数据添加到您的查询中。 请勿使用字符串插值来完成此操作,因为您将创建严重的SQL injection bugs。