雅可比矩阵的对角线（或散度）

Question

如何使用Pytorch高效计算雅可比矩阵的对角线？ 该算子广泛用于扩散模型中。

[d z_1/d x_1, d z_2/d x_2, ..., d z_n/d x_n]

一些不理想的替代方案是： 1. 先计算整个雅可比矩阵，然后取出对角线。 2. 遍历每个条目以单独计算导数。

Answer 1

解决方案：

import torch

# Define the function for which we want to compute the Jacobian
def func(x):
    y1 = x[0] ** 2 + x[1] ** 3
    y2 = x[1] ** 2 + x[0] ** 3
    return torch.stack([y1, y2])

# Define the input at which we want to evaluate the Jacobian
x = torch.tensor([1.0, 2.0], requires_grad=True)

# Compute the Jacobian of the function with respect to the input
J = torch.autograd.functional.jacobian(func, x)

print(J)

张量流解决方案：

import tensorflow as tf

# Define the function
def func(x):
    return tf.stack([tf.math.pow(x[0], 2) + tf.math.pow(x[1], 3), tf.math.pow(x[1], 2) + tf.math.pow(x[0], 3)])

# Define the inputs
x = tf.constant([1.0, 2.0])

# Create a GradientTape context to trace the operations
with tf.GradientTape() as tape:
    # Watch the input tensor
    tape.watch(x)
    # Evaluate the function
    y = func(x)

# Compute the Jacobian matrix of y with respect to x
jacobian = tape.jacobian(y, x)

print(jacobian)