Production planning model#

This tutorial illustrates how to build a simple optimal allocation model for production planning under resource constraints. The tutorial mirrors the workflow described in Model generation from scratch, and it is the right place to start if you are a CVXlab newbie.

Problem statement

Let us consider a production system, capable to produce two types of goods: \(product_1\) and \(product_2\), measured in \(units\). The system is characterized by the features listed below.

  • The production levels of the products are independent.

  • Production of each product generates constants profit generated, energy and material use per unit of output (i.e., no economies of scale or other non linearities are considered).

    \[\begin{split}\begin{array}{c|cc} & product_1 & product_2 \\ \hline c [€/u] & 1.0 & 2.2 \\ e [kWh/u] & 1.7 & 3.5 \\ m [kg/u] & 0.5 & 1.2 \end{array}\end{split}\]

The objective of the model is to determine the maximum-profit production plan, i.e. the quantity of each product produced leading to the overall system maximum profit, considering different scenarios for energy (\(E\)) and material (\(M\)) endowments.

\(E = (250, 300) kWh\), \(M = (60, 90) kg\)

In the zip directory, the materials related to this tutorial are available, including:

  • notebook.ipynb file: Jupyter Notebook with the complete workflow of the tutorial, including code and comments.

  • concept.xlsx file: Excel workbook with the conceptual model definition, resolving the model with the built-in solver. Useful to understand how the model works and the expected results.

  • model directory: model directory generated by CVXlab, containing: model settings (available in both xlsx and YAML formats), sets file sets.xlsx, SQLite file database.db.

The user can follow the step-by-step guide below, generating and filling the required files autonomoysly, or can run the provided Jupyter Notebook and rely on the materials provided.

Conceptual model definition#

Related user guide step: Conceptual model definition.

In this step, the model is formulated at a conceptual level, defining the Sets, Data Tables, Variables, and Problems (with related symbolic expressions). Notably, there is not one single way to define the model, and the same problem can be formulated in multiple equivalent ways.

Defining Sets

Sets defined for the model are summarized in the table below.

Sets of the tutorial model#

Set name

Symbol

Coordinates

Cardinality

Set type

Products

\(p\)

\(product_1\), \(product_2\)

2

Dimension set

Attributes

\(a\)

\(energy\), \(material\), \(profit\)

3

Dimension set

Scenarios_energy

\(e\)

\(low\), \(high\)

2

Inter-problem set

Scenarios_material

\(m\)

\(low\), \(high\)

2

Inter-problem set

Notice that:

  • Inter-problem sets (\(e\), \(m\)) defines multiple problem instances. This implies that one optimization problem is generated and solved for each combination of energy and material availability (in this case, \(2 \times 2 = 4\) problem instances will be generated).

  • Dimension sets (\(p\), \(a\)) are used to define the scope of Data Tables and the shapes of related variables.

  • Coordinates of each set can be associated to filters to define sub-domains. As example, different entries of the Attributes set \(a\) will be used to define different variables. In this simplified case, a filter will univocally used to identify each item of the set; however, in more complex models filters can be used to define overlapping sub-domains (a filter key may be associated to multiple coordinates).

Defining Data Tables and related Variables

The following tables summarizes the Data Tables and associated variables for the energy system model.

Data Tables of the tutorial model#

Name(sets domain)

Type

Domain [cardinality]

Description

\(production(p,e,m)\)

Endogenous

\(p \times e \times m \\ [2 \times 2 \times 2 = 8]\)

Total products output by energy and material availability (in units)

\(products_{data}(a,p)\)

Exogenous

\(a \times p \\ [3 \times 2 = 6]\)

Collects product attributes. Multiple variables will be defined based on this table, each related to a specific attribute.

\(energy_{avail}(e)\)

Exogenous

\(e [2]\)

Energy availability scenarios (in kWh)

\(material_{avail}(m)\)

Exogenous

\(m [2]\)

Material availability scenarios (in kg)

Notes:

  • Endogenous Data Table has a domain defined including all inter-problem sets, while exogenous Data Tables can be defined over specific inter-problem and dimension sets.

  • For each Data Table, the cardinality (i.e., row count) is reported, calculated as the product of the row counts of all sets in the domain. As example, the \(products_{data}\) Data Table includes 6 entries, representing the products attributes (3 entries: \(profit\), \(energy\), \(material\)) related to each product (2 entries: \(product_1\), \(product_2\)).

The variables associated to the Data Tables are summarized in the table below.

Variables of the tutorial model#

Source Data Table

Variable symbol

Shape [rows,cols]

Intra-problem sets

Inter-problem sets

Description

\(production(p,e,m)\)

\(x\)

\(1, p - [1,2]\)

\(-\)

\(e \times m - [4]\)

Products output (unit)

\(products_{data}(a,p)\)

\(c\)

\(1, p - [1,2]\)

\(-\)

\(-\)

Specific profit per product (€/unit)

\(products_{data}(a,p)\)

\(e\)

\(1, p - [1,2]\)

\(-\)

\(-\)

Specific energy use per product (kWh/unit)

\(products_{data}(a,p)\)

\(m\)

\(1, p - [1,2]\)

\(-\)

\(-\)

Specific material use per product (kg/unit)

\(energy_{avail}(e)\)

\(E\)

\(1, 1\)

\(-\)

\(e - [2]\)

Energy endowment (kWh)

\(material_{avail}(m)\)

\(M\)

\(1, 1\)

\(-\)

\(m - [2]\)

Material endowment (kg)

Regarding variables above:

  • Each variable stems from a related Data Table, inheriting its properties: among others, the domain (defined by sets) and the data type (exogenous, endogenous, constant).

  • Constants can be defined with different built-in or user defined types (see Constant data types). In the example above, only endogenous and exogenous variables are defined, while no constants are needed.

  • Notably, multiple variables can be associated to a same Data Table, each defined on a specific subset of the data table domain. In the example above, three variables \(c\), \(e\), and \(m\) are associated to the same \(products_{data}\) Data Table, each defined on a specific subset of the data table domain (these subsets are identified by filters on the Attributes set).

  • Each variable is characterized by a specific allocation of sets into dimensions sets. This allocation defines the shape of the variable (rows, columns) and the number of times the variable of that shape is repeated across the intra-problem sets domain (if any).

  • The cardinality of the inter-problem sets defines the number of times a variable is generated, corresponding to the number of problem instances. As example, the endogenous variable \(x\) is defined as a row vector of 2 columns (defined by the products set items), it is not indexed over any intra-problem coordinate, and it is indexed over 4 inter-problem coordinates (defined by the Cartesian product of the energy availability and material availability sets). Depending on the purpose of the variable in the numerical problem, it could be alternatively defined as a scalar variable, indexing products set as an inter-problem set instead of a shape set: in such case, a scalar variable is defined for each product, and the expressions where the variable is involved are each broadcasted over the product dimension.

Defining Problem and related Expressions

For the current energy system model, a symbolic problem can be defined as a system of linear inequalities reported below. The objective is to maximize the profit generated by the production plan, while respecting the energy and material constraints. A guard is added to ensure that the production levels are non-negative (the latter is not strictly required in this case, but it is a common practice in production planning models).

\[\begin{split}\begin{aligned} \max \quad & (c \cdot x') \\ \text{s.t.}\quad & e \cdot x' - E \leq 0 \\ & m \cdot x' - M \leq 0 \\ & x \geq 0 \end{aligned}\end{split}\]

Notice that:

  • The problem is defined a number of times equal to the cardinality of the inter-problem set. Specifically, one problem instance is defined and solved for each energy and material availability scenario. In this case, 4 problem instances are generated, corresponding to the Cartesian product of the energy availability and material availability sets.

  • For each simbolic expression, a number of numerical expressions is generated, equal to the Cartesian product of all intra-problem sets of the related variables. In this case, each symbolic expression is generated only 1 time, since no variable is indexed over intra-problem sets. In more complex models, the same symbolic expression can be generated multiple times, each time with different variable instances depending on the intra-problem sets domain.

  • In this problem, the dot operator \(\cdot\) represents matrix multiplication, the \((*)'\) represents the transposition operator (see Symbolic operators for a comprehensive description of built-in symbolic operators).

  • A note on dimensional formulations: the allocation of dimension sets to shapes and intra-problem sets offers significant modeling flexibility. The same problem can be formulated in multiple equivalent ways.

Generation of model directory#

Related user guide step: Generation of model directory
Related API: cvxlab.create_model_dir()

This step consists of creating the model directory scaffold that will later host the setup files, sets file, input data, database, and model results.

import cvxlab

cvxlab.create_model_dir(
    settings_file_type='yml',
)

Notice that:

  • If arguments model_dir_name and main_dir_path are not specified, the model directory is created with default name model in the current working directory.

  • The argument settings_file_type defines the format of the setup files. Both YAML and Excel formats are supported. The tutorial materials include setup files in both formats, so the tutorial can be followed regardless of the choice made at this stage.

  • At this stage, only the model directory and the setup file templates are generated. The sets file, input data file(s), and SQLite database are created in later steps.

    • Case of settings_file_type='yml': three YAML files are generated in the model directory, named structure_sets.yml, structure_variables.yml, and problem.yml.

    • Case of settings_file_type='xlsx': a single Excel workbook named model_settings.xlsx is generated in the model directory, with three tabs named structure_sets, structure_variables, and problem.

Fill model setup file(s)#

Related user guide step: Fill model setup file(s)

In this step, the conceptual model defined above is translated into CVXlab setup files. The same information can be written either in YAML format, using separate files, or in Excel format, using the model_settings.xlsx workbook.

For clarity, the examples below include only the fields required for this tutorial. Optional fields such as value, blank_fill, nonneg, copy_from, and aggregations are omitted because they are not needed in this model. This does not cause validation errors during setup parsing.

Defining model sets

This part defines the Sets used by the model. At this stage, only the Sets structure is defined; the actual set coordinates are entered later in sets.xlsx.

File: structure_sets.yml

Products:
  description: products | dimension set

Attributes:
  description: attributes of the products | dimension set
  filters:
    type: [profit, energy, material]

Scenarios_energy:
  description: energy availability scenarios | inter-problem set
  split_problem: True

Scenarios_material:
  description: material availability scenarios | inter-problem set
  split_problem: True

Defining model Data Tables and Variables

This part defines the Data Tables and the Variables generated from them.

File: structure_variables.yml

production:
  description: products supply
  type: endogenous
  coordinates: [Products, Scenarios_energy, Scenarios_material]
  variables_info:
    x:
      Products:
        dim: cols

products_data:
  description: attributes of products | profits, energy use, material use
  type: exogenous
  coordinates: [Products, Attributes]
  variables_info:
    c:
      Products:
        dim: cols
      Attributes:
        dim: rows
        filters: {type: profit}
    e:
      Products:
        dim: cols
      Attributes:
        dim: rows
        filters: {type: energy}
    m:
      Products:
        dim: cols
      Attributes:
        dim: rows
        filters: {type: material}

energy_avail:
  description: availability scenarios for energy
  type: exogenous
  coordinates: [Scenarios_energy]
  variables_info:
    E:

material_avail:
  description: availability scenarios for material
  type: exogenous
  coordinates: [Scenarios_material]
  variables_info:
    M:

Defining model symbolic problem

This part defines the objective function and the symbolic constraint expressions.

File: problem.yml

objective: Maximize(c @ tran(x))
expressions:
  - e @ tran(x) - E <= 0
  - m @ tran(x) - M <= 0
  - x >= 0
description:
  - maximization of total profit
  - energy use less than endowment
  - material use less than endowment
  - positive products supply

Model class instance generation#

Related user guide step: Model class instance generation
Related class constructor: cvxlab.Model

In this step, a Model instance is created. This is the main CVXlab object for handling subsequent operations, including data initialization, numerical problem generation, and solution.

import cvxlab

model = cvxlab.Model(
    log_level='debug',
    model_settings_from='yml', # or 'xlsx'
    use_existing_data=False,
    multiple_input_files=False,
    input_data_files_type="xlsx", # or "csv"
    detailed_validation=True,
)

The behavior of the constructor depends primarily on the argument use_existing_data:

  • If use_existing_data=False, the constructor parses the setup files and validates the model directory and setup files, loads the model structure, and generates a blank sets.xlsx file ready to be filled with model coordinates in the next step. This is the mode used when building a model from scratch.

  • If use_existing_data=True, in addition to the setup files parsing, the constructor also checks that the required model data already exist (sets.xlsx, the SQLite database, and the input-data directory), loads model coordinates, and initializes the numerical problem(s). In this case, the user can directly move to Solution of numerical problem(s).

In this tutorial, the use_existing_data argument must be set to False, because the model is being assembled step by step from empty templates. At the end of this step, the main output is a blank sets.xlsx file in the model directory, ready to be filled with set coordinates. The SQLite database and the input data file(s) are not generated yet; they are created in the next initialization step.

Notes:

  • If arguments model_dir_name and main_dir_path are not specified, the constructor looks for a model directory named model in the current working directory.

  • Log level and log format can be defined with the related arguments. The tutorial materials use log level debug, so the reader can inspect the internal operations performed by CVXlab during subsequent steps.

  • The argument model_settings_from defines the format of the setup files. Both YAML and Excel formats are supported, and must be coherently specified according to the setup file(s).

  • The argument multiple_input_files defines whether input data are collected in a single file or multiple files when those files are generated later. If False, exogenous data are written to one Excel workbook with separate tabs. If True, one file is generated per data table. In that case, input_data_files_type can also be set to "csv". In this tutorial, a single Excel input file is used.

  • The argument detailed_validation is useful while defining a model from scratch, because it provides more explicit feedback when the setup files contain incomplete or inconsistent definitions of Sets, Data Tables, Variables, or Problems.

Fill sets data (model coordinates)#

Related user guide step: Fill sets data (model coordinates)

In this step, the user manually fills the model coordinates in the Excel file sets.xlsx, which is placed in the model root by default. The workbook structure is automatically generated from the Sets definition provided in the setup file(s).

In this example, the workbook contains four tabs, one for each set defined in the model. Each tab is named _set_<SET_NAME>, where <SET_NAME> is the uppercase set key. These tab names are generated automatically by CVXlab and should not be changed.

Within each tab, the user fills only the coordinate values and, when present, the filter columns generated by CVXlab. Column headers are generated automatically and should not be edited.

In the _set_PRODUCTS tab, the user fills the coordinates of the Products set, i.e., the two products of the model: \(product_1\) and \(product_2\). No other columns are needed, since no filters nor aggregations are defined for this set.

sets.xlsx file | tab _set_PRODUCTS.#

Products_Name

product_1

product_2

In the _set_ATTRIBUTES tab, the user fills the coordinates of the Attributes set, i.e., the three attributes of the model: \(profit\), \(energy\), and \(material\). In this case, a type filter has been defined in the setup file(s), so the user also fills the second column with the filter value associated with each coordinate. In this simplified example, each filter value is associated with exactly one coordinate. In more complex models, the same filter value can be associated with multiple coordinates to define overlapping sub-domains.

sets.xlsx file | tab _set_ATTRIBUTES.#

Attributes_Name

Attributes_type

profit

profit

energy

energy

material

material

In the _set_SCENARIOS_ENERGY and _set_SCENARIOS_MATERIAL tabs, the user fills the coordinates of the Scenarios_energy and Scenarios_material sets, i.e., the energy and material availability scenarios of the model. Considering that these sets are defined here as inter-problem sets and no filters or aggregations are used for them, only the coordinates column is filled by the user.

sets.xlsx file | tab _set_SCENARIOS_ENERGY.#

Scenarios_energy_Name

low

high
sets.xlsx file | tab _set_SCENARIOS_MATERIAL.#

Scenarios_material_Name

low

high

Initialization of data structures#

Related user guide step: Initialization of data structures
Related API: initialize_model_environment()

In this step, the model environment is initialized from the filled sets.xlsx file. CVXlab loads the set coordinates into the model, then generates the blank SQLite database and the blank exogenous input data file(s).

model.initialize_model_environment()

The code block generates a blank SQLite database (named database.db by default) and the exogenous input data file(s) (in the <model_dir_name>/input_data directory by default) according to the structure defined in the setup file(s). The SQLite sets tables are filled with the coordinates defined in the sets file, while input data file(s) are generated blank, ready to be filled by the user with the numerical values of the exogenous data in the next step.

Fill exogenous model data#

Related user guide step: Fill exogenous model data

In this step, the user fills the numerical values of the exogenous data in the input data file(s) generated in the previous step. In this tutorial, a single Excel workbook named input_data.xlsx is generated, with one tab for each exogenous Data Table.

The user fills the exogenous data values in the values column, while the other columns (including the id and coordinate columns) are not edited, since they are automatically generated by CVXlab based on the sets coordinates and the Data Table domain defined in the setup file(s).

In this tutorial, the user fills three tabs as shown below, using the numerical data defined by the problem statement.

input_data.xlsx file | tab products_data.#

id

Products_Name

Attributes_Name

values

1

product_1

profit

1.0

2

product_1

energy

1.0

3

product_1

material

1.7

4

product_2

profit

2.2

5

product_2

energy

2.2

6

product_2

material

3.5
input_data.xlsx file | tab energy_avail.#

id

Scenarios_energy_Name

values

1

low

250

2

high

300
input_data.xlsx file | tab material_avail.#

id

Scenarios_material_Name

values

1

low

60

2

high

90

Initialization of numerical problem(s)#

Related user guide step: Initialization of numerical problem(s)
Related API: refresh_database_and_initialize_problem()

In this step, the numerical problem(s) are initialized. CVXlab imports the exogenous data from the input data file(s) into the SQLite database, validates the symbolic problem definition, creates the CVXPY variables, and generates the corresponding CVXPY Problem instances.

model.refresh_database_and_initialize_problem()

In this tutorial, four problem instances are generated, corresponding to the Cartesian product of the energy availability and material availability sets (\(2 \times 2 = 4\)). After this step, the numerical problems are ready to be solved.

Solution of numerical problem(s)#

Related user guide step: Solution of numerical problem(s)
Related API: run_model()

In this step, the CVXPY numerical problem(s) stored in the Model instance are solved. Although run_model() supports several configuration arguments, the default behavior is sufficient for this tutorial, so the user can simply run the code block below.

model.run_model()

During the solution process, CVXlab logs the main operations performed, including the selected solution mode, solver information, the status of each problem instance, and the execution time of the main steps. In this tutorial, all four problem instances are solved successfully and return the optimal status.

The log output can be useful to verify that the workflow completed successfully. For the tutorial narrative, however, the key result of this step is simply that the model has been solved and the endogenous values are now available in memory.

Export endogenous model data#

Related user guide step: Export endogenous model data
Related API: load_results_to_database()

In this step, the endogenous results computed by CVXlab are exported from the in-memory CVXPY variables to the SQLite database.

model.load_results_to_database()

After this step, the endogenous data tables in database.db contain the solved values for all scenarios. This makes the model results available for inspection, comparison, or external analysis tools.

Inspecting the exported results

It is useful to inspect the exported endogenous table directly, to verify that the model solved as expected and to interpret the production plan. In this tutorial, the exported endogenous results are stored in the table production.

Results from database.db | production table.#

Energy scenario

Material scenario

Product 1 output

Product 2 output

Active constraint(s)

low

low

0.000

42.857

energy

low

high

0.000

42.857

energy

high

low

120.000

0.000

material

high

high

155.172

10.345

energy and material

The results are economically consistent with the input coefficients. When energy is scarce (the two low energy scenarios), the model allocates all production to product_2, which yields the highest profit per unit of energy \((2.2 / 3.5 > 1.0 / 1.7)\). In these cases, the material limit is not binding, so increasing material availability alone does not change the optimal solution.

When energy availability is high but material availability is low (high/low), the limiting factor switches from energy to material. The model then allocates all production to product_1, which yields the highest profit per unit of material \((1.0 / 0.5 > 2.2 / 1.2)\).

In the high/high scenario, both resources are sufficiently available to support a mixed production plan. The solution combines product_1 and product_2 and fully exploits both the energy and material endowments, yielding the highest profit among the four scenarios. This is a useful final check of the tutorial workflow: the exported SQLite results are not only present, but also consistent with the underlying optimization logic.