Recent Releases of ieatools
ieatools - v0.1.77
IEATools 0.1.77 (2025-05-14)
- New function
reallocate_statistical_differences()reallocates statistical differences in proportion to the non-zero consumption of each energy carrier in other industries. extend_to_useful()now produces Ueioufu_details matrices with the correct row and column name structure when there is no energy industry own use.- New tests for new features.
- Now at 1385 tests, all passing.
- Test coverage remains at 100%.
- R
Published by MatthewHeun 10 months ago
ieatools - v0.1.76
IEATools 0.1.76 (2024-12-09)
- New function
load_electricity_heat_output()creates a data frame of IEA electricity and heat output information. - Change names of columns to avoid PostgreSQL database conflicts. PostgreSQL doesn't like column names containing "." because "." is the separator for "schema.table".
- New
specify_distribution_losses()function to specify distribution industries, within which transportation and distribution losses occur. - Argument added to
add_nuclear_industry()to ascribe some EIOU to the nuclear industry according to its output share. - New
specify_renewable_plants()function to specify renewable energy plants. - New
specify_electricity_grid()function to add an electricity grid. - The
.iea_fileargument toslurp_iea_to_raw_df()is now vectorized, which will enable sending a vector of country IEA data files in.iea_file. - Fixed several tests for new column names and added
a couple new tests for new features.
- Now at 1365 tests, all passing.
- Test coverage remains at 100%.
- R
Published by MatthewHeun about 1 year ago
ieatools - v0.1.75
IEATools 0.1.75 (2024-02-03)
extend_to_useful_helper()andextend_to_useful()now return additional matrices, namelyY_fu_detailsandU_EIOU_fu_details. These new matrices provide detailed information about the process of extending to the useful stage: the final energy product, the destination sector, the useful energy product, and the final-to-useful machine.- New tests for new features.
- Now at 1264 tests, all passing.
- Test coverage remains at 100%.
- R
Published by MatthewHeun about 2 years ago
ieatools - v0.1.73
IEATools 0.1.73 (2023-12-09)
- Attempt to fix Readme page on GitHub.
- Better error messages when empty allocations are encountered. Now each problem is shown on its own line.
- No new tests.
- Now at 1251 tests, all passing.
- Test coverage remains at 100%.
- R
Published by MatthewHeun about 2 years ago
ieatools - v0.1.72
IEATools 0.1.72 (2023-12-05)
- Fix RUS and EST Heat. The breakup of the Soviet Union (SUN) caused many irregularities in the IEA's energy accounting for SUN, Russia (RUS), and other former-Soviet states. In particular, the Flow of Heat is assigned to "Final consumption not elsewhere specified" and "Industry not elsewhere specified" for 1990--1992 (RUS) and 1990--1993 (EST). However, for following years, Heat is assigned to specific sectors. Other FoSUN countries do not exhibit the same problem. This Heat accounting irregularity is one of a series of problems that causes a jump in final-to-useful efficiencies for Europe and World in the CL-PFU database in the time period 1989--1990. This release includes code to perform a fix wherein Heat for both Final consumption not elsewhere specified and Industry not elsewhere specified is re-assigned to specific sectors by the proportion found in 1993 (RUS) and 1994 (EST) for 1990--1992 (RUS) and 1990--1993 (EST).
- New tests for new features.
- Now at 1251 tests, all passing.
- Test coverage remains at 100%.
- R
Published by MatthewHeun about 2 years ago
ieatools - v0.1.70
IEATools 0.1.70 (2023-11-06)
- Fix Australia Blast furnace gas.
Australia's Blast furnaces have an undesirable characteristic
that leads to singular matrices:
From 2013 onward, the production of Blast furnace gas by Blast furnaces
is consumed only by Blast furnaces.
No other industry or energy production machine consumes
Blast furnace gas.
In fact, the problem is deeper,
starting in 2010, the Iron and steel industry consumes no Blast furnace gas,
in apparent contradiction to the IEA's own policies for reporting
Blast furnace gas consumption.
fix_AUS_bfg()fixes the Blast furnace gas data for Australia for the 2010--2020 timeframe. - New tests for new features.
- Now up to 1241 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 2 years ago
ieatools - v0.1.69
IEATools 0.1.69 (2023-11-03)
- The constant
nonenergy_useis now aligned with IEA definitions for Non-energy use products. Specifically, now only "Additives/blending components", "Bitumen", "Lubricants", "Naphtha", "Paraffin waxes", "Refinery feedstocks", and "White spirit & SBP" are Non-energy use Products. "Coal tar", "Crude/NGL/feedstocks (if no detail)", "Crude oil", "Natural gas liquids", "Oil shale and oil sands", "Other hydrocarbons", and "Other oil products" were formerly included in the constant but have now been removed. - Added a fix for Other non-OECD Americas Gas works. For a few years (1971--1976), Other non-OECD Americas Gas works plants consume no feedstock. Fixed data are now included by default.
- Added a fix for Other non-OECD Americas Charcoal production plants. For many years (1971--2010), they produced Charcoal without consuming any energy (Primary solid biofuels is typical). The fixed data both (a) has Charcoal production plants consume Primary solid biofuels and (b) boosts production of Primary solid biofuels accordingly.
- New tests for new features.
- Now up to 1236 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 2 years ago
ieatools - v0.1.68
IEATools 0.1.68 (2023-08-18)
- Added note about permission to use IEA data to README.Rmd file.
- Zenodo DOI now used in references.
- Moved some internal constants to function arguments (
etwmbandetwab) inaggregate_regions(). - No new tests.
- Still at 1216 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 2 years ago
ieatools - v0.1.65
IEATools 0.1.65 (2023-04-27)
- Added new values to
IEATools::fap_flowsto accommodate detailed "Non-energy use in xxxxx" flows. - Fixing IEA 2022 release Electricity data for Colombia 1971--1977. The IEA acknowledged energy balance errors in those years. The fix is to use 2021 release data for 1971--1977 for both Colombia and World.
- No longer fixing Ghana's industrial consumption of electricity data, because details are now available in the IEA's WEEB data.
- Now using terajoules (TJ) as the preferred unit for all IEA extended energy balance data.
- Dropped support for 2018--2020 IEA extended energy balance data.
- Now using the 2022 release of IEA extended energy balance data by default.
- Functions now treat a zero-row incoming data frame much better, returning a zero-row data frame with columns of same type as would have been produced if the incoming data frame had at least one row.
IEATools::fd_sectorsnow contains specific "Non-energy use in <>" strings to support the option to specify Non-energy use flows when possible. - Fixed a bug where
"Stock changes [of Gas/diesel oil excl. biofuels]"
became
"Stock changes [of Gas/diesel oil excl]", because
the
notationargument was not being set properly in a call toRCLabels::get_pref_suff()when extracting prefixes. - Simplifications reduced the number of tests.
- Now up to 1216 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun almost 3 years ago
ieatools - v0.1.64
IEATools 0.1.64 (2023-03-09)
- New boolean argument
specify_non_energy_flowsonload_tidy_iea_df()enables specifying Non-energy use flows via the newspecify_non_energy_use()function where possible. For now, the default isFALSEto maintain backward compatibility (i.e., not specifying Non-energy use flows). - New function
specify_non_energy_use()uses "Memo: Non-energy use in <>" where possible. augment_iea_df()now addsFlow.aggregation.points for all aggregation flows that will (eventually) be removed. This change will enable future specification of "Non-energy use in xxxxx" flows with "Memo: Non-energy use in xxxxx" where possible.- Added capability to use
Matrixobjects withprep_psut(),form_C_mats(),form_eta_fu_phi_u_vecs(),extract_S_units_from_tidy(), andcollapse_to_tidy_psut(). This change enables sparse matrices to save memory and disk space. - Eliminated all warnings from the
tidyselectpackage about deprecated functionality. The package builds and tests cleanly again! - New argument on
add_psut_matnames()(R_includes_all_exogenous_flows) switches between (a) including all exogenous flows ("Resources", "Imports", "Statistical differences", and "Stock changes") in the R matrix (TRUE) and (b) placing only Resource flows in the R matrix (FALSE). Default isTRUE. - Added "Main activity producer electricity plants" to the list of
eiou_flows. We route "Own use in electricity, CHP and heat plants" to "Main activity producer electricity plants", so we need to include "Main activity producer electricity plants" in the EIOU flows. Same for "Natural gas extraction". - "Liquefaction (LNG) / regasification plants" EIOU flow now directed to the "Natural gas extraction" industry.
- New 4-letter abbreviations for non-standard ISO country codes.
- New tests for new features.
- Now up to 1371 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun almost 3 years ago
ieatools - v0.1.63
IEATools 0.1.63 (2022-04-01)
- Added
non_energy_flowstofd_sectorsto conform to IEA approach to calculating final energy. - Adding constant "nonenergy_products" which includes final energy products such as "Lubricants" which should only occur in the non-energy use sectors, but also occur in the total final consumption sectors.
- Responded to changes in
RCLabels. Now using "pref" and "suff" instead of "prefix" and "suffix" in several places. - New argument on
slurp_iea_to_raw_df(),ensure_ascii_countries, which converts characters with diacritic marks to straight ascii. The default value isTRUE. - Latin-1 encoding for IEA data files
now handled correctly in
slurp_iea_to_raw_df(). This change enables country names like "Cote d'Ivoire" and "Curacao/Netherlands Antilles" to be read correctly from the data file. - Adapt to changes in
RCLabels:keep_pref_suff()-->get_pref_suff(), among others. - Fixed a bug where Products with no supply would create an
NAand lead to a "Missing value where TRUE/FALSE needed" error. - Fixed a bug where countries with no Energy industry own use would fail in the workflow.
- R
Published by MatthewHeun almost 4 years ago
ieatools - v0.1.62
IEATools 0.1.62 (2021-10-12)
form_eta_fu_phi_u_vecs()now gives only the product name as the row name of the phi vectors, but it does so only after checking that all phi values are the same. If different phi values are detected for the same combination of metadata parameters and year, an error is emitted. This change solves a downstream issue where phi values from thephi_constants.xlsxfile have only products as row names, leading to incompatibility when summing phi vectors from the two sources and when multiplying the phi vectors into matrices where names of a dimension are product names only.form_eta_fu_phi_u_vecs()now changes the name of the single column of the exergy-to-energy ratio vector (phi_u) from "phi.u" to simply "phi", to become compatible with thephi_pfvector, whose single column is named "phi". This change fixes a downstream bug in theSEAPSUTWorkflowpackage.- Added functions to assist converting energy to exergy
when data are in PSUT matrix format.
New functions include:
load_phi_constants_table()sample_phi_constants_path()
- Many new tests for new features.
- Now up to 1087 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 4 years ago
ieatools - v0.1.61
- Added descriptive error messages when unknown
Flow.aggregation.point,Flow, orProductare encountered in arrangeieafuallocationtemplate()`. - Added new entries to the
fap_flowsconstant. These entries enable generation of templates for "World marine bunkers" and "World aviation bunkers". - Many new tests for new features.
- Now up to 1083 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 4 years ago
ieatools - v0.1.60
IEATools 0.1.60 (2021-09-06)
- No longer discriminating by sign when summing EIOU after routing Pumped storage plants and Own use in electricity, CHP, and heat plants. This change avoids a situation where positive electricity EIOU by Pumped storage plants in Japan caused a problem when writing the allocation template.
- Better error messages when energy imbalance is too large to fix.
- Expanded many tests to verify initialization functions work for all valid years of IEA sample data.
- Now able to read IEA Extended Energy Balance from 2020 and 2021 releases.
- Many new tests to cover all valid years of IEA data.
- Up to 1073 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 4 years ago
ieatools - v0.1.59
load_tidy_iea_df()has new arguments which are passed touse_iso_coutries(), thereby exposing the new functionality ofuse_iso_coutries()toload_tidy_iea_df().- New feature for
use_iso_countries(): can supply a data frame of ISO 3-letter codes in argumentoverride_df. - Fixed a test coverage failure:
There was no case where Ceioumat was missing.
when testing
extend_to_useful(). - A few new tests for new features.
- Up to 836 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 4 years ago
ieatools - v0.1.56
IEATools 0.1.56 (2021-08-18)
extend_to_useful()now robust to cases where there is no EIOU, thanks to now usingmatsindf::matsindf_apply().- Now using
matsindf::matsindf_apply()forextend_to_useful(). This change will enable better handling of cases where a country has no EIOU. The first use case will be bunkers. - No longer cleaning matrices in
extend_to_useful_helper(), because some vectors may be the0vector and be eliminated. - Added
replace_null_UR()function to the workflow in theprep_psut()function. - New function
replace_null_UR()replaces missing orNULLR,U_feed,U_EIOU,U, andr_EIOUwith 0 matrices with appropriate row and column names. The replacements are built from Y and V matrices. The need for this functionality arises when the last stage is final energy and imports (V matrix) are the only source of an energy carrier that is consumed in final demand (Y matrix). In that situation, the R and U matrices will be missing, and they can be replaced by 0 matrices with the right dimensions. - In the specification process,
the
Flowfor countriesWorld_X_bunkersare now specified to be "International navigation" and "International aviation" to correspond with their domestic equivalents. - New tests for new code.
- Up to 833 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 4 years ago
ieatools - v0.1.55
IEATools 0.1.55 (2021-07-21)
- Modifications to code and tests to accommodate World marine bunkers and World aviation bunkers being their own country.
- New tests for new features.
- Up to 808 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 4 years ago
ieatools - v0.1.53
IEATools 0.1.53 (2021-06-08)
- The "Oil and gas extraction" activity is split in "Oil extraction", which extracts oil products, and "Natural gas extraction", which extracts natural gas.
- New
split_oil_gas_extraction_eiou()function splits the EIOU of the "Oil and gas extraction" industry into EIOU for the "Oil extraction" and for the "Natural gas extraction" activities.
- R
Published by earamendia over 4 years ago
ieatools - v0.1.52
IEATools 0.1.52 (2021-05-07)
- Deleted an efficiency for LPG stoves in Ghana in 1971 that should not have been in the GH-ZA-Efficiency-sample-2019.xlsx file.
- Moved ABdatafull2018formattesting.csv to
inst/extdata. specify_primary_production()updated so that all products coming from Resources are now called Product [from Resources]. Then, for each product, a manufacturing industry takes these Product [from Resources] as inputs and transforms them into actual Products.
- R
Published by earamendia almost 5 years ago
ieatools - v0.1.50
IEATools 0.1.50 (2021-03-31)
- Now more robust support for tidy (instead of wide-by-year) format efficiency data.
- Now more arguments are passed from top-level functions to lower-level functions, thereby allowing external callers to use non-default column names.
- Moving some code to new syntax for referring to column names passed as function arguments.
- New tests for new features.
- Now up to 766 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun almost 5 years ago
ieatools - v0.1.47
IEATools 0.1.47 (2021-02-16)
- New
specify_all()function coded, so that it now splits non-specified industries, splits the IEA "Own use in electricity, CHP and heat plants" flow in main activity producer electricity, heat, and CHP plants, and so that a nuclear industry is added when needed. - Sub-functions for the specifyall() function are called within the `specifytp_eiou()` function.
- These include the
gather_producer_autoproducer(), theroute_pumped_storage(), theroute_own_use_elect_chp_heat(), theadd_nuclear_industry(), and theroute_non_specified_flows()functions. *specify_primary_production()function changed, the product naming is now modified. extract_S_units_from_tidy()function hotfixed so that it also works when the.tidy_iea_dfdata frame already has amatnamescolumn.
- R
Published by earamendia about 5 years ago
ieatools - v0.1.46
IEATools v0.1.46 (2020-02-01)
calc_tidy_iea_df_balancesfunction updated so that it also accounts for balancing flows when checking balances.- New bug in
aggregate_regions()hotfixed. - New option for "Epsilon" matrices and
PSUT columns enables removal of some
Flows for footprinting analyses (and maybe other work in the future). - When
matnamesis present in.tidy_iea_df,add_psut_matnames()now returns.tidy_iea_dfunmodified. This new feature enables other functions for assigning matrix names. - When all of
rownames,colnames,rowtypes, andcoltypesare present in.tidy_iea_df,add_row_col_meta()now returns.tidy_iea_dfunmodified. This new feature enables other functions for assigning row and column names and row and column types. - New tests for the new feature.
- Now up to 675 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by earamendia about 5 years ago
ieatools - v0.1.44
IEATools 0.1.44 (2020-12-08)
- Moved
primary_aggregates_IEA()andfinaldemand_aggregates_IEA()from theReccapackage toIEATools. Functions renamed to remove the_IEAsuffix. Hosting the functions inReccamust have made sense at some point. But now it is clear these functions should be inIEATools, because they aggregate IEA-style tidy data frames. - New tests for the new feature.
- Now up to 661 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun about 5 years ago
ieatools - v0.1.40
IEATools 0.1.40 (2020-11-23)
- Now setting row name of column sums in
extend_to_useful_helper()to get around a bug that occurs when a 1x1 matrix is column summed. - The API of
extend_to_useful()now assumes that columnsC_Y,C_eiou, andeta_fuare present in the incoming.sutdatadata frame. This change makes the API ofextend_to_useful()more consistent with the APIs of other functions in the package. - Still at 656 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.39
IEATools 0.1.39 (2020-11-19)
- Fixed a dormant bug in
form_eta_fu_phi_u_vecs()exposed by new code. form_eta_fu_phi_u_vecs()now accepts tidy data frames on input. This change allows better integration with theSEAPSUTWorkflowpackage.form_C_mats()now accepts tidy data frames on input. This change allows better integration with theSEAPSUTWorkflowpackage.- Still at 656 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.38
IEATools 0.1.38 (2020-11-19)
prep_psut()now returns a column for theUmatrix in addition toU_feedandU_EIOUcolumns.- Completed the first draft of a vignette on moving from final energy to useful energy at the last stage of an energy conversion chain.
- Still at 656 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 5 years ago
ieatools - IEATools 0.1.37 (2020-10-07)
- New
aggregate_regions()function that enables to aggregate regions of a.tidy_iea_dfbased on a user-defined aggregation table. - Underlying
read_aggregation_region_table()anddefault_aggregation_region_table_path()to enable theaggregate_regions()function to work. - Default IEA to Exiobase mapping provided as .xlsx file in the extdata folder.
- Now 656 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by earamendia over 5 years ago
ieatools - v0.1.36
IEATools 0.1.36 (2020-09-10)
- Fixed bug where missing etafu info caused failure.
Needed to allow for an empty `etafutable
incompleteetafutable()`. - New tests for the fixed bug.
- Now at 636 tests, all passing.
- Test coverage remains at 100 %.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.35
IEATools 0.1.35 (2020-08-26)
- First version that works with full SEAPSUT workflow.
- Removed an erroneous check for
energy industry own use in
specify_primary_production(). In fact, we need to specify all cases, not only if energy industry own use is present. This change solves a bug in adrakeworkflow. - Added spell checking to build process.
- R
Published by MatthewHeun over 5 years ago
ieatools - 0.1.33
IEATools 0.1.33 (2020-08-17)
- New tests for new features and old bugs.
- Now at 634 tests, all passing.
- Test coverage remains at 100 %.
- Change from
U_excl_EIOUtoU_feedeverywhere. fix_tidy_iea_df_balances()now deals correctly with a no-row IEA data frame.- Fixed a potential bug when deleting the
.errcolumn in fixtidyieadfbalances. - Fixed bugs with completion code.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.32
IEATools 0.1.32 (2020-03-13)
eta_fu_template()now accepts tidy data frames.- Adjusted tests for errors instead of warnings.
- Now at 611 tests, all passing.
- Test coverage remains at 100 %.
- New function
check_fu_allocation_data()verifies that allEf.productandEu.productentries are identical whenMachineisNon-energy.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.31
IEATools 0.1.31 (2020-07-29)
- Adjusted tests for errors instead of warnings.
- Now at 610 tests, all passing.
- Test coverage remains at 100 %.
.tolon allocation sums bumped from1e-10to1e-9, because some errors are3e-10.- Warnings are now errors with much more descriptive information.
- R
Published by MatthewHeun over 5 years ago
ieatools - 0.1.30
IEATools 0.1.30 (2020-07-24)
- New tests for new features and old bugs.
- Now up to 611 tests, all passing.
- Test coverage remains at 100 %.
- Fixed a bug where a completed final-to-useful allocation table
would have the
C_sourcecolumn, which caused a problem when checking for completion of a final-to-useful efficiency table. which_quantityargument tocomplete_eta_fu_table()now checked for validity.- Added "vignettes/References.bib" file. References now display properly on readme page.
- New function
tidy_eta_fu_table()does what the name suggests. This function (and previous new functiontidy_fu_allocation_table()) enable elimination of duplicated code in several places.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.29
IEATools 0.1.29 (2020-07-21)
- New tests for new features and old bugs.
- Now up to 606 tests, all passing.
- Test coverage remains at 100%.
extract_S_units_from_tidy()now correctly sets rowtype by default. rowtype wasUnit, but it should have beenProduct.- New function
tidy_fu_allocation_table()makes an FU allocation table tidy if it is not. year_cols()now returns the "Year" column, if it exists.- New function
meta_cols()gives metadata columns in any IEA data frame. - Refactored
complete_fu_allocation_table()andcomplete_eta_fu_table()to usefu_allocation_table_completed()andeta_fu_table_completed(). - New function
eta_fu_table_completed()that tells when an FU Efficiency table is complete. - New function
fu_allocation_table_completed()that tells when an FU Allocation table is complete. - Improvements to
switch-notationdocumentation. - Added constant
IEATools::fu_analysis_file_info.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.27
IEATools 0.1.27 (2020-06-29)
- New tests for new features.
- Now up to 567 tests, all passing.
- Test coverage remains at 100%.
- Added new function
remove_agg_regions()that removes known aggregation regions from an IEA extended energy balances data frame.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.26
IEATools 0.1.26 (2020-06-27)
- New tests for new features.
- Now up to 561 tests, all passing.
- Test coverage remains at 100%.
- Added functions to complete an FU Allocation table and an etaFU table:
`completefuallocationtable()
andcompleteetafu_table()`. - When generating an efficiency template with
eta_fu_template(), any Machine whose name begins with "Non-energy use" is now given default efficiency (eta.fu) of 0.0001% (i.e., 1e-6) and phi of 1.0. - Non-energy use no longer excluded when generating FU Allocation templates.
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.24
IEATools 0.1.24 (2020-06-17)
- Fixed a bug where "China (P.R. of China and Hong Kong, China)"
was not being coded to "CHN" in
use_iso_countries(). The problem is that thecountrycodepackage uses a different name from the IEA extended energy balance database. For now, I have coded "China (P.R. of China and Hong Kong, China)" as "CHN". Another option would be to code it as "CHNHKG".
- R
Published by MatthewHeun over 5 years ago
ieatools - v0.1.20
IEATools 0.1.20 (2020-05-01)
- Added additional tests for bug fixes and new features.
- Now up to 500 tests, all passing.
- Test coverage remains at 100%.
- Added function
move_last_stage_to_useful()and supporting functionsform_C_mats()andform_eta_fu_phi_u_vecs(). These functions use a matrix method to move from last stage of final energy to last stage of useful energy. - Changed default number of rows in an allocation template from 3 to 4,
per request from Zeke Marshall.
Note that the number of rows in the allocaiton template is adjustable
at the time the template is created using the
n_allocation_rowsargument tofu_allocation_template(). - Added functions for row and column notation in lists:
switch_notation_byname(),arrow_to_paren_byname(), andparen_to_arrow_byname(). These functions are like*_bynamefunctions in the packagematsbyname: they accept both single matrices and lists of matrices. In that way, they are amenable to pipeline calculations using thematsindfpackage. Adding this feature exposed bugs inmatsbyname::setrownames_byname()andmatsbyname::setcolnames_byname(). The bugs inmatsbynamehave been fixed, so these features are now working. - Added functions for row and column notation:
paren_to_arrow(),arrow_to_paren(), andswitch_notation().
- R
Published by MatthewHeun almost 6 years ago
ieatools - v0.1.19
IEATools 0.1.19 (2020-04-01)
- New function
form_eta_fu_phi_u_vecs()which creates column vectors of final-to-useful efficiencies (etafu) and exergy-to-energy ratios (phiu) from an efficiency table. - Added new test to hit code path for generating diagnostic information when FU Allocation table is mal-formed. Code coverage is back up to 100% for testing.
- R
Published by MatthewHeun almost 6 years ago
ieatools - v0.1.16
IEATools 0.1.16 (2020-03-24)
- Now up to 444 tests, which all pass.
iea_file_OK()now works on the 2018 IEA extended energy balance data.sort_iea_df()now works for specified IEA data frames. Internally,sort_iea_df()now usesdespecify_col().- New function
despecify_col()removes specification decoration from a column. - Now using prepositions in specification strings.
- Specification strings are now given in a package constant (
specify_notation). These constants are referred throughout the package. - All constants are now lists of character strings
instead of vectors to allow
$referencing of specific list items. - Added names to all constants for easier programming.
- R
Published by MatthewHeun almost 6 years ago
ieatools - v0.1.15
IEATools 0.1.15 (2020-03-16)
- Now up to 429 tests!
- Added a
fix_iea_datavignette. - Did preparatory work for future
fix_*functions, including the internaldo_fix()function which will work for fixing data from any country in any years, so long as a replacement data frame is available. - Added function
fix_GHA_industry_electricity()which adds specificity to Ghana's industry electricity. - Added function
replace_join()to assist with IEA data fixes. - Added named
iea_colsobject to simplify and standardize names for columns in IEA data frames, both tidy and wide. - Added a "Sorting" section to the IEATools vignette.
It describes the
sort_iea_df()function. - renamed
sort_tidy_iea_df()-->sort_iea_df(), because the function also sorts wide data frames. sort_tidy_iea_df()previously worked only with tidy data frames (with a column of years).sort_tidy_iea_df()now also works with wide data frames (with years spread to the right).- Fixed a bug in
sort_tidy_iea_df()that results inNAvalues in theLast.stagecolumn. - Added a function called
sort_tidy_iea_df()which sorts tidy IEA data frames with default IEA row orders. - Completed function called
fix_GHA_psb()which smooths Ghana's Primary solid biofuels data in the years 1991--1999.
- R
Published by MatthewHeun almost 6 years ago
ieatools - v0.1.14
IEATools 0.1.13 (2020-03-11)
- Fixed an issue with argument names: overwrite --> overwritefile.
The change was made in the
IEAToolspackage develop branch against which the latest version of the PFU-Database repository was built. However, the argument name change needs to be on the master branch ofIEAToolsso that `installgithub()` gets the updated argument name. - Added stub functions to fix certain IEA data. This capability is in development and not yet working.
- Added a script to make internal data frames for fixing IEA data. This capability is in development and not yet working.
- R
Published by MatthewHeun almost 6 years ago
ieatools - v0.1.13
IEATools 0.1.13 (2020-03-08)
- Added template tests for both 2018 and 2019.
- Added new sample allocation and F-->U efficiency tables for 2019 data.
- Rebuilt the sample template for F-->U allocation, because new categories of energy consumption were provided for ZA EIOU.
- Wrote new tests for the new approaches to determining "Transformation processes" and "Energy industry own use".
- Implemented a new way to identify "Transformation processes" and "Energy industry own use" rows.
The new method works for both the 2018 and 2019 editions of the IEA data.
The new method is based on finding breakpoints between sections of the table.
The method relies on the fact that each country's rows are in exactly the same order.
The validity of this assumption can be tested with
iea_file_OK(). - Added a new function
adjacent_rownums()that identifies the row numbers where two adjacent entries exist. This function is used extensively when determining the row ranges for "Transformation processes" and "Energy industry own use". - Added function
sample_iea_data_path()which takes a year (representing year of data release) as an argument. Default year is (now) 2019.
In subsequent years, I will change the default year to the most-recent year available. 2018 is also supported. This function allows supporting and testing of previous years' data versions. - Added
iea_file_OK()which tells whether all countries in an IEA extended energy balance table have the same order for all rows. - Added "Paper, pulp and printing" to the list of industries for 2019. (In the 2018 edition of the IEA's data, "Paper, pulp and print" was the name of this industry.)
- There is no longer an "Other" aggregation in the IEA database. One test that had expected an "Other" aggregation point in the database (using 2018 data) was failing, because the expectation is no longer met in the 2019 version of the database. The test has been changed to no longer expect an "Other" aggregation point.
- Focused some tests that assumed some FLOWs would end in
"(transf.)", "(transformation)", or "(energy)" on sample IEA data from 2018.
No FLOWs end with those strings in the 2019 data. There is no harm to leave the code that strips trailing "(transf.)", "(transformation)", and "(energy)". augment_iea_df()now recognizes "Final consumption not elsewhere specified" as an "other" flow. "Final consumption not elsewhere specified" (in the 2019 data) is a synonym for "Non-specified (other)" (in 2018 and earlier editions of the IEA data).augment_iea_df()now recognizes "Industry not elsewhere specified" as an industry flow. "Industry not elsewhere specified" (in the 2019 data) is a synonym for "Non-specified (industry)" (in 2018 and earlier editions of the IEA data).iea_df()now removes white space, even from "FLOW" entries that are quoted to protect commas.
- R
Published by MatthewHeun almost 6 years ago
ieatools - v0.1.10
IEATools 0.1.10 (2020-01-22)
- Integrated with codecov.io by adding
codecov.ymlfile to repository. - Integrated with Travis by adding
.travis.ymlfile to repository. Now, Travis builds the package automatically after each commit. In addition, test coverage is calculated after each build. - Now at 100% test coverage.
Got to this point by adding new tests to ensure that all parts of the code are being hit.
And by adjusting the regex strings in
extract_TK().
- R
Published by MatthewHeun about 6 years ago
ieatools - v0.1.8
IEATools 0.1.8 (2019-07-24)
- Efficiency template now includes energy flows into final-to-useful machines for each year, indicating to analysts which years need no efficiency or exergy-to-energy ratio.
- New
sort_byargument foreta_fu_template()allows "usefulenergytype" and "importance". "usefulenergytype" sorts first bymd,light,ke, andheat, then by magnitude of energy flow into the machine. "importance" sorts by magnitude of energy flow into the machine only. Default is "usefulenergytype".
- R
Published by MatthewHeun over 6 years ago
ieatools - v0.1.7
- Allocation template now sorting rows in IEA order.
- All specified primary flows now consumed somewhere in the ECC.
- Now specifying the source of output flows from inserted machines
at primary level.
For example, inserting Coal mines into the ECC gives now results in
Hard coal produced by (Coal mines).
For example, "Hard coal (if no detail)" --> "Hard coal (if no detail) (Coal mines)". - Added
fu_allocation_template()function that makes templates for deciding allocations to final-to-useful machines in Final consumption or Energy industry own use. specify_tp_eiou()now aggregates inputs and outputs separately at the end of its specification process. Doing so avoids a situation where resource extraction industries would look like a Transformation process sink industry when inputsand outputs were netted together.
- R
Published by MatthewHeun almost 7 years ago
ieatools - v0.1.6
IEAData 0.1.6 (2019-04-06)
- Updated all three vignettes to reflect the new functions and demonstrate their use.
- New function
augment_iea_df()now removes(energy),(transf.), and(transformation)suffixes from theFlowcolumn after adding theFlow.aggregation.pointcolumn. - No longer removing countries that lack a 2-letter ISO abbreviation.
- Added
prep_psut()function to bundle several data munging steps into one. - Added
production_to_resources()function to convertProductiontoResources (product). - Added
specify_primary_production()function to fix primary production issues in the IEA data. - Added constants
coal_and_coal_products,oil_and_oil_products,renewable_products,biofuel_and_waste_products. - Rename package from
IEADatatoIEAToolsto better reflect its purpose. Users can't get data from this package, anyway. This package will do more than distribute data. This package allows users to also manipulate data and extend to useful stage. - Now includes sample data from 2 countries and 2 years.
- Converted to functions for all tasks so more general.
- Added function to create an S_units matrix from tidy data.
- R
Published by MatthewHeun almost 7 years ago