为什么 Cloud Function 在本地环境中运行 2 分钟而不是 2 秒？

Question

对 GCP 非常陌生，而不是 Python 专家，我尝试在云函数中运行脚本（在某些条件下在数据库中读取/写入），但在 GCP 中运行需要 2 分钟（即使是 2gb）与2s 在我的 Jupyter 笔记本中在本地环境中。 有人可以帮助我如何优化这个时间吗？我已经尝试以 2Gb 部署云功能，但运行时间相同〜 2 分钟 （我也已经读过这篇文章：Why google cloud function runs more that 2 minutes? 以及云功能的最佳实践，但不是那么容易理解！）

Cloud Function 中的脚本下方（主要函数为：main_binary_alert()）

import sqlalchemy
import pandas as pd
import pg8000

def load_data_from_db(sql_query):
    
    db=sqlalchemy.create_engine(
        "XX",
        pool_size=5,
        max_overflow=2,
        pool_timeout=30,
        pool_recycle=1800
    )
    
    try:
        with db.connect() as conn:
            df=pd.read_sql_query(sql_query,con=conn)
    except Exception as e:
        return 'Error: {}'.format(str(e))
    return df
 
def insert_data_into_db(sql_query):
    
    db=sqlalchemy.create_engine(
        "XX",
        pool_size=5,
        max_overflow=2,
        pool_timeout=30,
        pool_recycle=1800
    )
    
    stmt=sqlalchemy.text(sql_query)
    
    try:
        with db.connect() as conn:
            conn.execute(stmt)
    except Exception as e:
        return 'Error: {}'.format(str(e))
    return 'ok'

def check_condition(logic_condition,_to_check,check_):
    if logic_condition==None or check_==None:
        return True
    if _to_check==None:
        return False
    if logic_condition=='superieur':
        if _to_check > check_:
            return True
        else:
            return False
    if logic_condition=='superieur_ou_egal':
        if _to_check >= check_:
            return True
        else:
            return False
    if logic_condition=='inferieur':
        if _to_check < check_:
            return True
        else:
            return False
    if logic_condition=='inferieur_ou_egal':
        if _to_check <= check_:
            return True
        else:
            return False
 
def main_binary_alert(request):
 
    try:
        X=0
        alert_param=load_data_from_db("SELECT * FROM alert_param")
        alert_criteria_binary=[1,4,8,9,16,10,11,12,13,14,15]
    alert_param_filterA=alert_param.loc[(alert_param['alert_criteria_id'].isin(alert_criteria_binary)) & (alert_param['alert_mute']==False)]
 
        
        for row in alert_param_filterA.itertuples():
 
            alert_id=row.alert_id
            alert_location_id=row.alert_location_id
            value_type_id=row.value_type_id
            logic_condition_1=row.logic_condition_1
            _condition_1=row._condition_1
            logic_condition_2=row.logic_condition_2
            _condition_2=row._condition_2
 
            last_same_alert_in_x_day=load_data_from_db("SELECT * FROM alert_reading \
            WHERE alert_id="+str(alert_id)+"\
            AND time IN (SELECT MAX(time) FROM alert_reading WHERE alert_id="+str(alert_id)+"AND time >= date_trunc('day',now()-interval '"+str(X)+"day')) \
            AND time >= date_trunc('day',now()-interval '"+str(X)+"day')")    
 
            last_check_button=load_data_from_db("SELECT * FROM binary_reading_client1_raw \
                    WHERE time >= date_trunc('day',now()-interval '"+str(X)+"day') \
                    AND time IN (SELECT MAX(time) FROM binary_reading_client1_raw \
                        WHERE time >= date_trunc('day',now()-interval '"+str(X)+"day') \
                        AND value_id IN (SELECT value_id FROM value WHERE value_type_id=10 AND location_id="+str(alert_location_id)+")) \
                    AND value_id IN (SELECT value_id FROM value WHERE value_type_id=10 AND location_id="+str(alert_location_id)+")"
            )        
            
            if last_same_alert_in_x_day.empty==True:
 
                if last_check_button.empty==True:       
 
                    sum_from_X_days=load_data_from_db(
                    "SELECT SUM() FROM binary_reading_client1_raw \
                    WHERE time >= date_trunc('day',now()-interval '"+str(X)+"day')\
                    AND value_id IN (SELECT value_id FROM value WHERE value_type_id="+str(value_type_id)+" \
                    AND location_id="+str(alert_location_id)+")"
                    )
 
                    if check_condition(logic_condition_1,sum_from_X_days.iloc[0,0],_condition_1)==True & check_condition(logic_condition_2,sum_from_X_days.iloc[0,0],_condition_2)==True:
                        insert_data_into_db(
                            "INSERT INTO alert_reading (alert_id,time,location_id) S \
                            ("+str(alert_id)+",now(),"+str(alert_location_id)+")"
                        )
                        print('case: alert triggered based on counter from X days') 
                    else:
                        print('case:no alert triggered: no past same alert, no past check, no condition met') 
    
                else:
 
                    last_check_time=last_check_button.iloc[0,1]            
 
                    sum_from_last_check=load_data_from_db(
                    "SELECT SUM() FROM binary_reading_client1_raw \
                    WHERE time >= "+"'"+str(last_check_time)+"'"+" \
                    AND value_id IN (SELECT value_id FROM value WHERE value_type_id="+str(value_type_id)+" \
                    AND location_id="+str(alert_location_id)+")"
                    )
 
                    if check_condition(logic_condition_1,sum_from_last_check.iloc[0,0],_condition_1)==True & check_condition(logic_condition_2,sum_from_last_check.iloc[0,0],_condition_2)==True:
 
                        insert_data_into_db(
                            "INSERT INTO alert_reading (alert_id,time,location_id) S \
                            ("+str(alert_id)+",now(),"+str(alert_location_id)+")"
                        )
                        print('case: alert triggered based on counter from past time check') 
                    else:
                        print('case:no alert triggered: no past same alert, past check, no condition met') 
 
            else:
 
                last_alert_time=last_same_alert_in_x_day.iloc[0,2]
 
                if last_check_button.empty==True:
 
 
                    sum_from_last_alert=load_data_from_db(
                    "SELECT SUM() FROM binary_reading_client1_raw \
                    WHERE time >= "+"'"+str(last_alert_time)+"'"+" \
                    AND value_id IN (SELECT value_id FROM value WHERE value_type_id="+str(value_type_id)+" \
                    AND location_id="+str(alert_location_id)+")"
                    )            
 
                    if check_condition(logic_condition_1,sum_from_last_alert.iloc[0,0],_condition_1)==True & check_condition(logic_condition_2,sum_from_last_alert.iloc[0,0],_condition_2)==True:
                        #write an alert in alert_reading
                        insert_data_into_db(
                            "INSERT INTO alert_reading (alert_id,time,location_id) S \
                            ("+str(alert_id)+",now(),"+str(alert_location_id)+")"
                        )
                        print('case: alert triggered based on counter from past time alert') 
                    else:
                        print('case:no alert triggered: past same alert, no past check, no condition met') 
 
                else:
 
                    last_check_time=last_check_button.iloc[0,1] 
 
                    max_time_alert_check=max(last_alert_time,last_check_time)
 
                    sum_from_max_time_alert_check=load_data_from_db(
                    "SELECT SUM() FROM binary_reading_client1_raw \
                    WHERE time >= "+"'"+str(max_time_alert_check)+"'"+" \
                    AND value_id IN (SELECT value_id FROM value WHERE value_type_id="+str(value_type_id)+" \
                    AND location_id="+str(alert_location_id)+")"
                    )  
 
                    if check_condition(logic_condition_1,sum_from_max_time_alert_check.iloc[0,0],_condition_1)==True & check_condition(logic_condition_2,sum_from_max_time_alert_check.iloc[0,0],_condition_2)==True:
                        insert_data_into_db(
                            "INSERT INTO alert_reading (alert_id,time,location_id) S \
                            ("+str(alert_id)+",now(),"+str(alert_location_id)+")"
                        )
                        print('case: alert triggered based on counter from max time alert and check') 
                    else:
                        print('case:no alert triggered: past same alert, past check, no condition met')
        
        return ("ok", 200)
    except Exception as e:
        return 'Error: {}'.format(str(e))

Answer 1

函数是无状态的，执行环境通常是从头开始初始化的，这称为冷启动，这意味着您的云函数在一段时间内不运行时会变慢。增加的延迟是由于Cloud Functions 每次都会启动运行环境。

那么，您如何才能缩短响应时间。您所要做的就是让云功能按时间间隔运行。 Cronjobs 是为此类任务而设计的。由于您已经在 Google Cloud Platform 上运行，请查看他们的调度程序。您可以尝试使用this 参考执行

我每分钟创建一个云调度程序作业来执行云功能。

Frequency is * * * * * (means every minute).
Time Zone is US Newyork, the same region in which the cloud function was created(us-central1)
URL would be the cloud function’s endpoint and trigger URL. 
Auth Header has to be specified as an Add OIDC token and create a service account and give the role Cloud functions Invoker permissions to this service account.  
Audience is again cloud functions endpoint trigger URL.
Maximum retry attempts is 2 or anything as per your request
Max retry duration is 0s which means unlimited.

通过每分钟安排一个 cron 作业，我正在运行云功能，这样它就不必面对冷启动。如果一个函数在 10 分钟左右执行完，每次都会从头开始，需要更长的部署时间。

通过这种方法，Cloud Functions 将缓存此运行环境。因此，在缓存“热”时定期（每 1 分钟）运行不会导致启动损失。

注意：这可能会给您带来不必要的费用，因为每分钟安排一次功能可能会耗尽您的免费配额。

如需更多参考，请查看this 链接以了解 Cloud Functions 及其最佳实践。