SciPy Optimizer

SciPy'soptimizemodule provides implementations of common numerical optimization and equation-solving algorithms. We can directly call these functions to solve practical problems, such as finding the minimum or maximum of a function, or finding the roots of equations.

NumPy itself mainly focuses on array computation and linear algebra operations. It providespolynomial root-findingtools (such asnumpy.roots), but it does not provide a genericnonlinear equation numerical root-finding interface.

For example, the following nonlinear equation:

x + cos(x)

For such problems, you can use thescipy.optimize.rootfunction to solve it. The two core parameters commonly used by this function are as follows:

• fun- The function representing the equation (a return value of 0 indicates that the root has been found)
• x0- The initial guess for the root

rootThe function returns a result object (OptimizeResult), which contains information such as the solving status, whether it succeeded, the function value, and the final solution.

The actual solution is stored in thexattribute of the returned object. Example:

Executing the above code produces the following output:

[-0.73908513]

If you print the returned object directly, you can view more complete solution information:

Executing the above code produces the following output:

     fjac: array([[-1.]])
      fun: array([0.])
  message: 'The solution converged.'
     nfev: 9
      qtf: array([-2.66786593e-13])
        r: array([-1.67361202])
   status: 1
  success: True
        x: array([-0.73908513])

Where:

• successIndicates whether the algorithm converged successfully
• messageDescribes the result of the solving process
• xThe final root of the equation obtained

Minimization Functions

In mathematics and engineering problems, a function can usually be represented as a curve, and the curve often has high and low points.

The high points of a curve are calledmaxima。

The low points of a curve are calledminima。

The highest point on the entire curve is called theglobal maximum, while the other higher but not highest points are calledlocal maxima。

Similarly, the lowest point on the entire curve is called theglobal minimum, while the other lower but not lowest points are calledlocal minima。

In SciPy, you can use thescipy.optimize.minimize()function to find the minimum of a function.

minimize()The commonly used parameters of the function are described below:

• fun- The objective function to be minimized

• x0- The initial guess for the independent variable

• method- The name of the optimization algorithm used. Common optional values include:'CG'、'BFGS'、'Newton-CG'、 'L-BFGS-B'、'TNC'、 'COBYLA'、'SLSQP'

• callback- A callback function called after each iteration

• options- A dictionary of other control parameters, for example:

  {
      "disp": boolean,  # 是否打印详细的优化过程信息
      "gtol": number    # 梯度收敛容差（部分算法适用）
  }

Executing the above code produces the following output:

       fun: 1.75
  hess_inv: array([[0.50000001]])
       jac: array([0.])
   message: 'Optimization terminated successfully.'
      nfev: 8
       nit: 2
      njev: 4
    status: 0
   success: True
         x: array([-0.50000001])

From the result, we can see that the function, atx ≈ -0.5attains its minimum, and the corresponding minimum function value is1.75, which is consistent with the analytical result.

minimize()The function also supports constrained optimization problems. You can add constraints through theconstraintsparameter by specifying equality or inequality constraints, which is suitable for more complex practical application scenarios.

When the objective function cannot be differentiated directly, you can choosemethod='Nelder-Mead'`Nelder-Mead` or other optimization algorithms that do not require gradient information. Such methods gradually approach the optimal solution through simplex search.

In practice,successthe `success` field indicates whether the optimization converged successfully,funthe `fun` field is the optimal objective function value,xand the `x` field is the optimal solution.successIt is recommended to always check the `success` status to ensure the reliability of the solution.

Advanced Root Finding

In addition to theroot()function, SciPy also provides thefsolve()function for solving equation roots. It is more concise and suitable for quickly solving simple equations.

Executing the above code produces the following output:

[-0.73908513]

fsolve()It directly returns an array of solutions, whileroot()the latter returns a result object containing more information.fsolveThe former is more convenient.

Solving with Multiple Initial Values

For equations with multiple roots, starting from different initial values may yield different roots. The following solvesx² - 4 = 0the two roots:

Executing the above code produces the following output:

初始值 -3 -> 根: -2.000000
初始值  0 -> 根:  2.000000
初始值  3 -> 根:  2.000000

Solving Equations with Parameters

In practical problems, equations often contain parameters. We can pass extra parameters to the objective function through theargsparameter:

Executing the above code produces the following output:

解: -0.450184

Constrained Optimization

minimize()The function supports adding constraints, including equality constraints and inequality constraints. Here is an example of a constrained optimization problem:

Executing the above code produces the following output:

最优解: x=0.500000, y=0.500000
最小值: 0.500000

Constraint types'eq'Indicates an equality constraint (the constraint function must equal 0),'ineq'Indicates an inequality constraint (the constraint function must be greater than or equal to 0).

Bound-Constrained Optimization

In addition to functional constraints, you can also use theboundsparameter to limit the value range of the independent variable:

Executing the above code produces the following output:

最优解: x=1.000000
最小值: 2.000000

Gradient-Free Optimization

When the objective function is non-differentiable or its derivative is difficult to compute, you can use an optimization algorithm that does not require gradient information, such as theNelder-Meadmethod:

Executing the above code produces the following output:

最优解: x=-1.047198
最小值: -1.523599
是否成功: True

Common Issues and Suggestions

• Choose an appropriate initial value- The choice of initial value has a great impact on the solution result. It is recommended to perform a simple visualization or analysis of the objective function before solving.
• Check the convergence status- Always check thesuccess`success` field to ensure the algorithm has converged successfully. If solving fails, try changing the initial value or the optimization algorithm.
• Choose an appropriate algorithm- For unconstrained problems,BFGS`BFGS` is a good default choice; for large-dimensional problems, you can try `L-BFGS-B`;L-BFGS-Bfor non-differentiable functions, use `Nelder-Mead`.Nelder-Mead。
• Handling Multiple Roots- For problems with multiple roots, it is recommended to solve starting from multiple different initial values.
• Parameter Passing- Use theargs`args` parameter to pass extra arguments, avoiding the use of global variables.

Summary

SciPy'soptimizemodule provides powerful optimization and equation-solving tools:

• Root finding：root()andfsolve()for solving nonlinear equations
• Unconstrained optimization：minimize()combined with themethod`method` parameter to implement multiple algorithms
• Constrained optimization: via theconstraints`constraints` parameter to add equality and inequality constraints
• Bound constraints: useboundsParameter limits the range of variable values
• Gradient-free methods：Nelder-MeadApplicable to non-differentiable functions

After mastering these basic functions, readers can further exploreoptimizeMore features of the module, such as curve fitting (curve_fit), global optimization (basinhopping、differential_evolution) and other advanced features.