Note
Click here to download the full example code
Usage of the model-versus-data code verification tool on a Finite-Element (Abaqus) model¶
Check an Abaqus cantilever beam model versus a reference dataset with the 'CodeVerificationAgainstData' tool.
from __future__ import annotations
import logging
from gemseo.datasets.io_dataset import IODataset
from gemseo.utils.directory_creator import DirectoryNamingMethod
from vimseo import EXAMPLE_RUNS_DIR
from vimseo.api import activate_logger
from vimseo.api import create_model
from vimseo.core.model_settings import IntegratedModelSettings
from vimseo.tools.verification.verification_vs_data import CodeVerificationAgainstData
We first define the logger level:
activate_logger(level=logging.INFO)
Then we create the model to verify:
model_name = "BendingTestAnalytical"
load_case = "Cantilever"
model = create_model(
model_name,
load_case,
model_options=IntegratedModelSettings(
directory_archive_root=EXAMPLE_RUNS_DIR / "archive/verification_vs_data",
directory_scratch_root=EXAMPLE_RUNS_DIR / "scratch/verification_vs_data",
cache_file_path=EXAMPLE_RUNS_DIR
/ f"caches/verification_vs_data/{model_name}_{load_case}_cache.hdf",
),
)
Out:
INFO - 16:51:59: Found 2 entries in the cache file : /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/model_runs/caches/verification_vs_data/BendingTestAnalytical_Cantilever_cache.hdf node : node
We also need a reference dataset. Here we do it programmatically, but we can also create it from a csv file: ``
reference_data = IODataset().from_array(
data=[[10.0, 10.0, -4.0, -12.0], [15.0, 10.0, -6.0, -40.0]],
variable_names=["height", "width", "maximum_dplt", "reaction_forces"],
variable_names_to_group_names={
"height": "inputs",
"width": "inputs",
"maximum_dplt": "outputs",
"reaction_forces": "outputs",
},
)
All inputs to the verification are now available. We create the verification tool we are interested in.
verificator = CodeVerificationAgainstData(
directory_naming_method=DirectoryNamingMethod.NUMBERED,
working_directory="CodeVerificationAgainstData_results",
)
# The options can be modified.
# Alternatively, options can be passed as keyword arguments to
# ``CodeVerificationAgainstModelFromParameterSpace()`` constructor.
verificator.options["metric_names"] = [
"SquaredErrorMetric",
"RelativeErrorMetric",
"AbsoluteErrorMetric",
]
verificator.execute(
model=model,
reference_data=reference_data,
output_names=["maximum_dplt", "reaction_forces"],
description={
"title": "Verification of a cantilever analytic beam for a variation of beam height.",
"element_wise": ["Small height value", "High height value"],
},
)
Out:
INFO - 16:51:59: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/03_verification_vs_data/CodeVerificationAgainstData_results
INFO - 16:51:59: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/03_verification_vs_data/CodeVerificationAgainstData_results/CustomDOETool
INFO - 16:51:59: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/03_verification_vs_data/CodeVerificationAgainstData_results/CustomDOETool
INFO - 16:51:59:
INFO - 16:51:59: *** Start DOEScenario execution ***
INFO - 16:51:59: DOEScenario
INFO - 16:51:59: Disciplines: Model BendingTestAnalytical: An analytical model for the bending of a parallelepipedic beam
INFO - 16:51:59:
INFO - 16:51:59: Load case:
INFO - 16:51:59: Load case Cantilever: A cantilever load case.
INFO - 16:51:59:
INFO - 16:51:59: Boundary condition variables:
INFO - 16:51:59: ['imposed_dplt', 'relative_dplt_location']
INFO - 16:51:59:
INFO - 16:51:59: Plot parameters:
INFO - 16:51:59: {
INFO - 16:51:59: "curves": []
INFO - 16:51:59: }
INFO - 16:51:59: Load:
INFO - 16:51:59: Load(direction='', sign='', type='')
INFO - 16:51:59:
INFO - 16:51:59: Default values:
INFO - 16:51:59:
INFO - 16:51:59: Default geometrical variables:
INFO - 16:51:59: {"height": [40.0], "length": [600.0], "width": [30.0]}
INFO - 16:51:59:
INFO - 16:51:59: Default numerical variables:
INFO - 16:51:59: {}
INFO - 16:51:59:
INFO - 16:51:59: Default boundary conditions variables:
INFO - 16:51:59: {"imposed_dplt": [-5.0], "relative_dplt_location": [1.0]}
INFO - 16:51:59:
INFO - 16:51:59: Default material variables:
INFO - 16:51:59: {"nu_p": [0.3], "young_modulus": [210000.0]}
INFO - 16:51:59: model_inputs:
INFO - 16:51:59: [
INFO - 16:51:59: "length",
INFO - 16:51:59: "width",
INFO - 16:51:59: "height",
INFO - 16:51:59: "imposed_dplt",
INFO - 16:51:59: "relative_dplt_location",
INFO - 16:51:59: "young_modulus",
INFO - 16:51:59: "nu_p"
INFO - 16:51:59: ]
INFO - 16:51:59: model_outputs:
INFO - 16:51:59: [
INFO - 16:51:59: "reaction_forces",
INFO - 16:51:59: "maximum_dplt",
INFO - 16:51:59: "dplt_grid",
INFO - 16:51:59: "location_max_dplt",
INFO - 16:51:59: "dplt",
INFO - 16:51:59: "moment",
INFO - 16:51:59: "moment_grid",
INFO - 16:51:59: "dplt_at_force_location",
INFO - 16:51:59: "error_code",
INFO - 16:51:59: "model",
INFO - 16:51:59: "load_case",
INFO - 16:51:59: "description",
INFO - 16:51:59: "job_name",
INFO - 16:51:59: "persistent_result_files",
INFO - 16:51:59: "n_cpus",
INFO - 16:51:59: "date",
INFO - 16:51:59: "cpu_time",
INFO - 16:51:59: "user",
INFO - 16:51:59: "machine",
INFO - 16:51:59: "vims_git_version",
INFO - 16:51:59: "directory_archive_root",
INFO - 16:51:59: "directory_archive_job",
INFO - 16:51:59: "directory_scratch_root",
INFO - 16:51:59: "directory_scratch_job"
INFO - 16:51:59: ]
INFO - 16:51:59: MDO formulation: DisciplinaryOpt
INFO - 16:51:59: Optimization problem:
INFO - 16:51:59: minimize maximum_dplt(height, width)
INFO - 16:51:59: with respect to height, width
INFO - 16:51:59: over the design space:
INFO - 16:51:59: +--------+-------------+-------+-------------+-------+
INFO - 16:51:59: | Name | Lower bound | Value | Upper bound | Type |
INFO - 16:51:59: +--------+-------------+-------+-------------+-------+
INFO - 16:51:59: | height | -inf | None | inf | float |
INFO - 16:51:59: | width | -inf | None | inf | float |
INFO - 16:51:59: +--------+-------------+-------+-------------+-------+
INFO - 16:51:59: Solving optimization problem with algorithm CustomDOE:
INFO - 16:51:59: 50%|█████ | 1/2 [00:00<00:00, 34.34 it/sec, obj=-5]
INFO - 16:51:59: 100%|██████████| 2/2 [00:00<00:00, 39.65 it/sec, obj=-5]
INFO - 16:51:59: Optimization result:
INFO - 16:51:59: Optimizer info:
INFO - 16:51:59: Status: None
INFO - 16:51:59: Message: None
INFO - 16:51:59: Number of calls to the objective function by the optimizer: 2
INFO - 16:51:59: Solution:
INFO - 16:51:59: Objective: -5.0000000000000036
INFO - 16:51:59: Design space:
INFO - 16:51:59: +--------+-------------+-------+-------------+-------+
INFO - 16:51:59: | Name | Lower bound | Value | Upper bound | Type |
INFO - 16:51:59: +--------+-------------+-------+-------------+-------+
INFO - 16:51:59: | height | -inf | 10 | inf | float |
INFO - 16:51:59: | width | -inf | 10 | inf | float |
INFO - 16:51:59: +--------+-------------+-------+-------------+-------+
INFO - 16:51:59: *** End DOEScenario execution (time: 0:00:00.056774) ***
VerificationResult(metadata=ToolResultMetadata(generic={'datetime': '11-05-2026_16-51-59', 'version': '0.1.7.dev11+g45528c259'}, misc={}, settings={'input_names': [], 'output_names': ['maximum_dplt', 'reaction_forces'], 'metric_names': ['SquaredErrorMetric', 'RelativeErrorMetric', 'AbsoluteErrorMetric'], 'description': {'title': 'Verification of a cantilever analytic beam for a variation of beam height.', 'element_wise': ['Small height value', 'High height value']}}, report={}, model=ModelDescription(name='BendingTestAnalytical', summary=' An analytical model for the bending of a parallelepipedic beam', load_case=Beam_Cantilever(name='Cantilever', domain='Beam', summary='A cantilever load case.', plot_parameters=PlotParameters(curves=[]), bc_variable_names=['imposed_dplt', 'relative_dplt_location'], load=Load(direction='', sign='', type='')), dataflow={'model_inputs': ['length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'model_outputs': ['reaction_forces', 'maximum_dplt', 'dplt_grid', 'location_max_dplt', 'dplt', 'moment', 'moment_grid', 'dplt_at_force_location', 'error_code', 'model', 'load_case', 'description', 'job_name', 'persistent_result_files', 'n_cpus', 'date', 'cpu_time', 'user', 'machine', 'vims_git_version', 'directory_archive_root', 'directory_archive_job', 'directory_scratch_root', 'directory_scratch_job'], 'PreBendingTestAnalytical_Cantilever': {'inputs': ['length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'outputs': ['imposed_dplt_location', 'quadratic_moment', 'reaction_forces', 'moment', 'moment_grid', 'solver', 'boundary']}, 'RunBendingTestAnalytical': {'inputs': ['imposed_dplt_location', 'quadratic_moment', 'reaction_forces', 'moment', 'moment_grid', 'solver', 'boundary', 'length', 'width', 'height', 'imposed_dplt', 'relative_dplt_location', 'young_modulus', 'nu_p'], 'outputs': ['dplt', 'dplt_grid', 'moment', 'moment_grid', 'imposed_dplt_location', 'reaction_forces']}, 'PostBendingTestAnalytical_Cantilever': {'inputs': ['dplt', 'dplt_grid', 'moment', 'moment_grid', 'imposed_dplt_location', 'reaction_forces'], 'outputs': ['reaction_forces', 'maximum_dplt', 'dplt_grid', 'location_max_dplt', 'dplt', 'moment', 'moment_grid', 'dplt_at_force_location', 'error_code']}, 'subroutine_names': []}, default_inputs={<InputGroupNames.NUMERICAL_VARS: 'numerical variables'>: {}, <InputGroupNames.BC_VARS: 'boundary conditions variables'>: {'imposed_dplt': [-5.0], 'relative_dplt_location': [1.0]}, <InputGroupNames.GEOMETRICAL_VARS: 'geometrical variables'>: {'length': [600.0], 'width': [30.0], 'height': [40.0]}, <InputGroupNames.MATERIAL_VARS: 'material variables'>: {'young_modulus': [210000.0], 'nu_p': [0.3]}}, curves=[('dplt_grid', 'dplt'), ('moment_grid', 'moment')], verbose=False)), simulation_and_reference=GROUP inputs outputs Reference
VARIABLE height width cpu_time maximum_dplt reaction_forces maximum_dplt reaction_forces
COMPONENT 0 0 0 0 0 0 0
0 10.0 10.0 0.021409 -5.0 -12.152778 -4.0 -12.0
1 15.0 10.0 0.003780 -5.0 -41.015625 -6.0 -40.0, element_wise_metrics=GROUP SquaredErrorMetric RelativeErrorMetric ... AbsoluteErrorMetric inputs
VARIABLE maximum_dplt reaction_forces maximum_dplt ... reaction_forces height width
COMPONENT 0 0 0 ... 0 0 0
0 1.0 0.023341 0.250000 ... 0.152778 10.0 10.0
1 1.0 1.031494 0.166667 ... 1.015625 15.0 10.0
[2 rows x 8 columns], integrated_metrics={'SquaredErrorMetric': {'maximum_dplt': 1.0, 'reaction_forces': 0.5274175950038581}, 'RelativeErrorMetric': {'maximum_dplt': 0.2083333332881946, 'reaction_forces': 0.019061053239892913}, 'AbsoluteErrorMetric': {'maximum_dplt': 1.0, 'reaction_forces': 0.5842013888888893}}, description={'title': 'Verification of a cantilever analytic beam for a variation of beam height.', 'element_wise': ['Small height value', 'High height value']})
The result contains the error metrics:
verificator.result.integrated_metrics
Out:
{'SquaredErrorMetric': {'maximum_dplt': 1.0, 'reaction_forces': 0.5274175950038581}, 'RelativeErrorMetric': {'maximum_dplt': 0.2083333332881946, 'reaction_forces': 0.019061053239892913}, 'AbsoluteErrorMetric': {'maximum_dplt': 1.0, 'reaction_forces': 0.5842013888888893}}
And saved on disk, together with its metadata:
verificator.save_results()
Out:
INFO - 16:51:59: Saving result to /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/03_verification_vs_data/CodeVerificationAgainstData_results/CodeVerificationAgainstData_result.hdf5
INFO - 16:51:59: Saving result to /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/03_verification_vs_data/CodeVerificationAgainstData_results/CustomDOETool/CustomDOETool_result.hdf5
The saved results can be loaded in a dedicated dashboard to be explored.
The dashboard is opened by typing dashboard_verification in a terminal,
and selecting the tab Comparison case.
The results can also be plotted from the Python API. It shows the scatter matrix of the inputs:
figures = verificator.plot_results(
verificator.result,
"RelativeErrorMetric",
"reaction_forces",
save=False,
show=True,
directory_path=verificator.working_directory,
)
Out:
/home/sebastien.bocquet/PycharmProjects/vimseo/.tox/doc/lib/python3.11/site-packages/pandas/plotting/_matplotlib/misc.py:100: UserWarning:
Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
/home/sebastien.bocquet/PycharmProjects/vimseo/.tox/doc/lib/python3.11/site-packages/pandas/plotting/_matplotlib/misc.py:101: UserWarning:
Attempting to set identical low and high ylims makes transformation singular; automatically expanding.
/home/sebastien.bocquet/PycharmProjects/vimseo/.tox/doc/lib/python3.11/site-packages/pandas/plotting/_matplotlib/misc.py:91: UserWarning:
Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
/home/sebastien.bocquet/PycharmProjects/vimseo/.tox/doc/lib/python3.11/site-packages/gemseo/utils/matplotlib_figure.py:59: UserWarning:
FigureCanvasAgg is non-interactive, and thus cannot be shown
INFO - 16:51:59: Working directory is /home/sebastien.bocquet/PycharmProjects/vimseo/docs/runnable_examples/03_verification_vs_data/CodeVerificationAgainstData_results
and an histogram of the errors:
figures["error_metric_histogram"]
Total running time of the script: ( 0 minutes 1.059 seconds)
Download Python source code: plot_bending_test_vs_data.py