deforce: Derivative-Free Algorithms for Optimizing Cascade Forward Neural Networks

deforce (DErivative Free Optimization foR Cascade forward nEural networks) is a Python library that implements variants and the traditional version of Cascade Forward Neural Networks. These include Derivative Free-optimized CFN models (such as genetic algorithm, particle swarm optimization, whale optimization algorithm, teaching learning optimization, differential evolution, ...) and Gradient Descent-optimized CFN models (such as stochastic gradient descent, Adam optimizer, Adelta optimizer, ...). It provides a comprehensive list of optimizers for training CFN models and is also compatible with the Scikit-Learn library. With deforce, you can perform searches and hyperparameter tuning for traditional CFN networks using the features provided by the Scikit-Learn library.

• Free software: GNU General Public License (GPL) V3 license
• Provided estimator: CfnRegressor, CfnClassifier, DfoCfnRegressor, DfoCfnClassifier, DfoTuneCfn
• Total DFO-based CFN models: > 200 regressors, > 200 classifiers.
• Total GD-based CFN models: 12 regressors, 12 classifiers.
• Supported performance metrics: >= 67 (47 regressions and 20 classifications)
• Supported objective functions: >= 67 (47 regressions and 20 classifications)
• Documentation: https://deforce.readthedocs.io
• Python versions: >= 3.8.x
• Dependencies: numpy, scipy, scikit-learn, pandas, mealpy, permetrics, torch, skorch

Citation Request

If you want to understand how to use Derivative Free-optimized Cascade Forward Neural Network, you need to read the paper titled "Optimization of neural-network model using a meta-heuristic algorithm for the estimation of dynamic Poisson’s ratio of selected rock types". The paper can be accessed at the following link

Please include these citations if you plan to use this library:

```bibtex @article{van2024deforce, title={deforce: Derivative-free algorithms for optimizing Cascade Forward Neural Networks}, author={Van Thieu, Nguyen and Nguyen, Hoang and Garg, Harish and Sirbiladze, Gia}, journal={Software Impacts}, volume={21}, pages={100675}, year={2024}, publisher={Elsevier}, doi={10.1016/j.simpa.2024.100675}, url={https://doi.org/10.1016/j.simpa.2024.100675} }

@software{thieudeforce2024, author = {Van Thieu, Nguyen}, title = {{deforce: Derivative-Free Algorithms for Optimizing Cascade Forward Neural Networks}}, url = {https://github.com/thieu1995/deforce}, doi = {10.5281/zenodo.10935437}, year = {2024} }

@article{van2023mealpy, title={MEALPY: An open-source library for latest meta-heuristic algorithms in Python}, author={Van Thieu, Nguyen and Mirjalili, Seyedali}, journal={Journal of Systems Architecture}, year={2023}, publisher={Elsevier}, doi={10.1016/j.sysarc.2023.102871} } ```

Installation

• Install the current PyPI release: sh $ pip install deforce

After installation, check the installed version by:

```sh $ python

import deforce deforce.version ```

Examples

Please check documentation website and examples folder.

1) deforce provides this useful classes

python from deforce import DataTransformer, Data from deforce import CfnRegressor, CfnClassifier from deforce import DfoCfnRegressor, DfoCfnClassifier

2) What can you do with all model classes

```python from deforce import CfnRegressor, CfnClassifier, DfoCfnRegressor, DfoCfnClassifier

Use standard CFN model for regression problem

regressor = CfnRegressor(hiddensize=50, act1name="tanh", act2name="sigmoid", objname="MSE", maxepochs=1000, batchsize=32, optimizer="SGD", optimizer_paras=None, verbose=False, seed=42)

Use standard CFN model for classification problem

classifier = CfnClassifier(hiddensize=50, act1name="tanh", act2name="sigmoid", objname="NLLL", maxepochs=1000, batchsize=32, optimizer="SGD", optimizer_paras=None, verbose=False, seed=42)

Use Metaheuristic-optimized CFN model for regression problem

print(DfoCfnClassifier.SUPPORTEDOPTIMIZERS) print(DfoCfnClassifier.SUPPORTEDREG_OBJECTIVES)

optparas = {"name": "WOA", "epoch": 100, "popsize": 30} regressor = DfoCfnRegressor(hiddensize=50, act1name="tanh", act2name="sigmoid", objname="MSE", optimizer="OriginalWOA", optimizerparas=optparas, verbose=True, seed=42)

Use Metaheuristic-optimized CFN model for classification problem

print(DfoCfnClassifier.SUPPORTEDOPTIMIZERS) print(DfoCfnClassifier.SUPPORTEDCLS_OBJECTIVES)

optparas = {"name": "WOA", "epoch": 100, "popsize": 30} classifier = DfoCfnClassifier(hiddensize=50, act1name="tanh", act2name="softmax", objname="CEL", optimizer="OriginalWOA", optimizerparas=optparas, verbose=True, seed=42) ```

3) After you define the model, do something with it + Use provides functions to train, predict, and evaluate model

```python from deforce import CfnRegressor, Data

data = Data() # Assumption that you have provide this object like above

model = CfnRegressor(hiddensize=50, act1name="tanh", act2name="sigmoid", objname="MSE", maxepochs=1000, batchsize=32, optimizer="SGD", optimizer_paras=None, verbose=False)

Train the model

model.fit(data.Xtrain, data.ytrain)

Predicting a new result

ypred = model.predict(data.Xtest)

Calculate metrics using score or scores functions.

print(model.score(data.Xtest, data.ytest, method="MAE")) print(model.scores(data.Xtest, data.ytest, list_methods=["MAPE", "NNSE", "KGE", "MASE", "R2", "R", "R2S"]))

Calculate metrics using evaluate function

print(model.evaluate(data.ytest, ypred, list_metrics=("MSE", "RMSE", "MAPE", "NSE")))

Save performance metrics to csv file

model.saveevaluationmetrics(data.ytest, ypred, listmetrics=("RMSE", "MAE"), savepath="history", filename="metrics.csv")

Save training loss to csv file

model.savetrainingloss(save_path="history", filename="loss.csv")

Save predicted label

model.saveypredicted(X=data.Xtest, ytrue=data.ytest, savepath="history", filename="y_predicted.csv")

Save model

model.savemodel(savepath="history", filename="traditional_CFN.pkl")

Load model

trainedmodel = CfnRegressor.loadmodel(loadpath="history", filename="traditionalCFN.pkl") ```