什麼是旋度

 https://www.eet-china.com/mp/a123145.html

https://www.math.sinica.edu.tw/mathmedia/journals/4386

基於物理的機器學習

Here’s a table summarizing the analogy between the loss function in optimization and scalar potential in physics:

Aspect	Loss Function (Optimization)	Scalar Potential (Physics)
Definition	A scalar function $L(\theta)$ that measures the "error" or "cost" for a given set of parameters $\theta$.	A scalar field $V(\mathbf{r})$ that represents potential energy per unit charge or mass at a point in space.
Domain	Parameter space $\theta$ (e.g., weights of a neural network).	Physical space $\mathbf{r}$ (e.g., 3D spatial coordinates).
Gradient ($\nabla$)	The gradient $\nabla L(\theta)$ points in the direction of steepest increase in the loss.	The gradient $\nabla V(\mathbf{r})$ points in the direction of steepest increase in potential.
Negative Gradient ($-\nabla$)	$-\nabla L(\theta)$ gives the direction of steepest decrease in the loss.	$-\nabla V(\mathbf{r})$ gives the direction of steepest decrease in potential energy.
Physical Interpretation	Minimizing $L(\theta)$ adjusts parameters to improve model predictions (reduce error).	Systems naturally move to minimize potential energy, leading to equilibrium states.
Movement	Parameters $\theta$ are updated iteratively in the direction $-\nabla L(\theta)$ to minimize loss.	Particles "move downhill" in the direction $-\nabla V(\mathbf{r})$ to minimize potential energy.
Landscape	Loss function $L(\theta)$ forms a "loss landscape" with hills, valleys, and minima.	Potential $V(\mathbf{r})$ forms a "potential landscape" with similar features.
Goal	Find the global minimum of the loss $L(\theta)$.	Find the state of lowest potential energy $V(\mathbf{r})$.
Stochasticity	Stochastic gradient descent uses noisy estimates of $\nabla L(\theta)$, introducing randomness to avoid local minima.	Deterministic; no stochasticity in the potential field (but randomness can exist in other contexts, like thermal motion).
Applications	Optimization in machine learning (e.g., neural network training).	Describing forces in physical systems (e.g., gravitational or electric fields).