Proving Competency: Energy Audits for Investment Grade Certification
Popular energy efficiency certifications such as the CEM (Certified Energy Manager) and CEA (Certified Energy Auditor) can be obtained by attending a training course then sitting a multiple choice exam using a hand-held calculator.
The IGEA certification is different in that it requires you to prepare an energy audit of a real site.
Why?
Why should IGEA candidates go to the effort of finding a site and undertaking an energy audit of it? Isn’t this inconvenient? Certainly, it would be less time-consuming for candidates to sit a multiple-choice exam.
And why should IGEA candidates pay more than USD $100 to be certified? Surely if the exam was multiple choice, it would be inexpensive to grade, and the cost of the certification could be reduced!
And why do IGEA candidates need the person they have worked with at the site which was audited to vouch for them? And even to produce a geo-tagged photo of themselves with the site contact on the site! And why does the site contact need to participate in an interview where he/she is asked questions about the IGEA candidate? Surely this is an imposition on the site!
There is a one-word answer to these questions.
TRUST.
The core premise of the IGEA certification is that it should be trustworthy. That someone who is certified as an IGEA can, without supervision, undertake an investment-grade energy audit. That an energy user contracting an IGEA can be confident that the auditor can deliver a quality energy audit.
In the rest of this article, I am going to:
- Identify the competencies an investment grade energy auditor must have.
- Classify the skills required to be an energy auditor.
- Explain the types of evidence that are needed to establish competency as an investment-grade energy auditor.
- Explain why passing a multiple-choice exam does not provide adequate evidence of competency and how someone who has passed such an exam has not actually demonstrated that he/she is competent to undertake an energy audit.
- Demonstrate how the assessment of a real energy audit, assessed in a systematic way against the competencies of an investment-grade energy auditor, is trustworthy.
- Provide evidence of why trust is so important when it comes to energy auditor certification.
The competencies an investment grade energy auditor must have
By definition an “investment grade” energy audit is one in which the business case for investments in energy efficiency is trustworthy. The actual costs and benefits of investing in energy efficiency improvements should be in line with the amounts presented in the energy audit report.
In order to achieve this, an energy auditor needs to be competent in two main areas:
- Having the skill to be able to follow a systematic audit process that enables the identification and accurate quantification of energy efficiency opportunities.
- Having the knowledge of how energy using technologies on site use energy, and what the opportunities are to reduce energy usage.
In each of these two areas, there are many areas of skill and knowledge required.
Several of the key competencies required by energy auditors in the Australian and New Zealand Energy Audit standards for commercial buildings (AS3598:2014 Part 1) and industrial sites (AS3598:2014 Part 2) include:
- Collecting data (including fieldwork and measurements).
- Analysing the data collected for accuracy, completeness and relevance.
- Identification, reporting and analysis of systems problems and risks.
- Developing energy models of businesses… including energy end-use breakdowns.
- Preparing the audit report and recommendations.
- Competencies related to specific building or industrial systems, such as heating, ventilation and air conditioning, lighting, electric motors, etc.
- Knowledge related to specific building systems such as comfort criteria, illuminance levels and lighting quality, etc.
Classifying the skills required by an energy auditor
Skills in any area can be classified by using a taxonomy.
A taxonomy is simply a method of classification. For example, libraries use the Dewey Decimal System for classifying books. You may be familiar with this system. Under this taxonomy, a book with starting with the number 620 would be related to engineering and applied operations.
The Bloom taxonomy is a well-known taxonomy for classifying understanding of a particular area.
The Bloom taxonomy has six levels. At the lowest level of the Bloom taxonomy is “Remembering”, and at the highest level is “Creating.”
At each level various verbs can be used to classify the competency.
An Investment Grade Energy Auditor needs to be able to operate at that highest level of being able to create solutions that save energy, and that go beyond simple equipment replacement or turning loads off when they aren’t in use. In essence an energy audit is a new or original work.
By way of explanation, let’s just zoom in on two of the core abilities of energy auditors – being able to develop an energy-use breakdown, and being able to identify and quantity energy savings opportunities.
To develop an energy use breakdown you need to:
- collect data (such as meter data, and data on each load in the building – such as nameplate data and operating hours);
- understand what the data means (such as nameplate data);
- apply knowledge (such as understanding how the different loads use energy within the systems they operate in) to be able to estimate a load factor for each individual load in the facility;
- collate data for all loads and analyse and evaluate this by comparing it with the actual site total annual energy usage;
- assess the completeness and accuracy of your energy use breakdown and make adjustments to balance it with the actual site energy usage.
To be able to do this, an energy auditor needs to have the competence to be able to work right up to the 2nd highest level of the Bloom taxonomy – evaluate.
To be able to identify and quantify energy savings opportunities you need to:
- develop an energy use breakdown as described above, to be able to identify significant energy uses and be able to “sense-check” your savings estimates;
- apply knowledge of how energy can be saved including analysing systems
- create and design solutions to deliver savings that go well beyond simply replacing an item of equipment with an identical, but more efficient item of equipment.
In this case the energy auditor needs to be able to competent right up to the highest level of the Bloom taxonomy – to create.
For example, I recently worked with someone who isn’t yet an Investment Grade Energy Auditor, assessing savings opportunities in a hotel. He identified items such as improving the switch off of lights, or replacing compact fluorescent lights with LED lights. In doing this he showed he was able to collect data, understand what it meant, and apply some knowledge to come up with solutions. He was also able to analyse and quantify the savings opportunity.
He wasn’t capable, however, of being about to identify much larger savings opportunities that I could see. He could get up to the 4th, perhaps the 5th level of the taxonomy. But doesn’t yet have the skill needed to be an investment grade auditor.
He wasn’t able to identify the big savings opportunity possible by significantly changing (i.e. involving a re-design) of the domestic hot water system.
To be able to identify that opportunity, I drew on my knowledge and understanding of hot water systems, applied my knowledge of the physics and thermodynamics of delivering hot water to the hotel rooms, analysed the data that was available using regression analysis and some other techniques, evaluated the losses in the existing system, and created a new system design estimated to use around 40% less energy.
Evidence requirements for demonstrating competency
Back when I studied at the Australian Institute of Management to become a trainer, and subsequently became certified under the Australian Qualifications Framework, one of the three modules I studied was on how to assess competency.
I learnt that there are three main types of evidence of competency:
- Direct
- Indirect
- Supplementary
Direct evidence is the best form of evidence, which can include observing the candidate performing the task and analysis of real-life work documentation, such as an energy audit report.
Indirect evidence is based on the observations of a qualified third party (e.g. via written or verbal questioning), or an analysis of outcomes or outputs.
Supplementary evidence is the weakest form of evidence, and is obtained through written or oral questioning, written assignments or case studies, etc.
Why passing a multiple choice exam doesn’t demonstrate energy auditor competency
The Certified Energy Auditor (CEA), Certified Energy Manager (CEM), Certified Measurement and Verification Professional (CMVP), Performance Measurement and Verification Analyst (PMVA) and several other certifications in the energy efficiency field involve attending a training course then sitting an open-book exam that usually runs for 4 hours. The use of a hand held calculator is permitted.
What type of evidence do you think the candidate is providing when sitting a multiple choice exam?
Is it Direct, Indirect or Supplementary?
Clearly this is supplementary evidence. Supplementary evidence is the weakest form of evidence.
One of the key challenges energy auditors face is collecting and interpreting data. The site visit is an indispensable part of an energy audit.
Can a multiple choice exam assess an energy auditor’s ability to collect data and to know which data to collect?
No. In a multiple choice exam the data is provided. No effort required.
Energy auditors need strong data manipulation skills, and need to be expert in the use of spreadsheets. An investment grade auditor can create new solutions, which may mean creating new calculations. Energy auditing software can be useful, but generally has limited flexibility when it comes to applying it in novel ways. Spreadsheets give this flexibility.
An energy auditor should be able to analyse energy equipment inventories using pivot tables and analyse data and must be able to establish data relationships using regression analysis. Can this be done using a hand-held calculator with a multiple choice exam?
No. This is a core competency of an energy auditor, yet it cannot be tested with a multiple choice exam.
Can multiple choice exams test the ability to be able to create or design? Energy auditors must be able to create and design energy saving solutions. The biggest savings are often achieved by modifying or changing a system – essentially designing a new system.
Is it possible to design something new where you are presented with 4 solutions to choose from in a multiple choice question?
No, it is totally impossible. You don’t have to create anything. You just have to choose one of the 4 multiple choice options.
A successful audit is one in which the client moves ahead and implements recommendations in the energy audit report. A relationship of trust is established with the client.
Is it possible to test, using a multiple choice exam, whether or not a candidate has the skill to be able to build a relationship of trust with an energy user?
No.
Since 2015, through my association with the Efficiency Valuation Organisation (EVO), where I am the chair of the training committee, and with the Energy Efficiency Council (EEC) – a training partner of EVO, I have delivered training on the fundamentals of Measurement and Verification (M&V) of energy savings. This training, up until the start of 2022 led into the CMVP exam, and now leads into the PMVA exam. Both of these are multiple choice exams.
Interestingly, however, it has been recognized by EVO, that the multiple choice exam is not all that effective in determining if a candidate is capable of developing a M&V plan that is adherent with the International Performance Measurement and Verification Protocol (IPMVP).
Consequently EVO has developed the Performance Measurement and Verification Expert (PMVE) certification which does away with the assessment being based solely on a multiple choice exam. Much of the PMVE assessment is based on candidates being able to develop core elements of an M&V plan.
So it can be seen that trusted organisations, such as EVO, are also recognizing the weakness of multiple choice exams for assessing energy efficiency related competencies.
Why the Investment Grade Energy Auditor assessment of competency is trustworthy
An energy audit report is direct evidence of a candidate’s ability to undertake an energy audit. The candidate has had to engage with the site, undertake a site visit, collect and analyse data, produce a report, present it to the energy user and build a relationship of trust with the energy user, such that the energy user moves ahead with the energy auditor’s recommendations.
As direct evidence, it is the strongest form of evidence.
The assessment of the investment grade energy audit prepared by the candidate is a comprehensive assessment, with a focus on identifying on whether or not the candidate is capable of designing solutions that aren’t just a like-for-like equipment replacement. The candidate is being assessed to verify if the candidate has reached the highest level of the Bloom taxonomy – the ability to create something new and original.
By interviewing the site contact nominated by the candidate, the assessment gets indirect evidence of the candidates ability to build a relationship of trust with the energy user.
The IGEA certification requires the candidate to create something from nothing. The candidate is not told what site to audit, given no data sets, not presented with a set of multiple choice questions. To be able to do this the candidate has to draw on multiple areas of knowledge and skill.
The energy audit then submitted by the candidate is carefully analysed against a list of core competencies in a systematic way by a highly experienced energy auditor.
As a result, if a candidate is assessed as competent by the assessor, the candidate has truly demonstrated that he/she can be trusted to undertake an investment grade energy audit.
Why is trust so important when it comes to energy auditor certification?
An energy auditor could be considered to be akin to an investment advisor when it comes to energy efficiency. An energy auditor will commonly advise clients to spend tens or hundreds of thousands of dollars on energy efficiency improvements.
With energy efficiency considered to be the first fuel, when it comes to saving money on energy bills, and on reducing greenhouse gas emissions, many businesses invest in energy efficiency. Total global expenditure on energy efficiency upgrades in 2022 reached USD 560 billion in 2022 according to the International Energy Agency. I don’t know how much of this investment was based on energy auditor advice, but its likely to have been a sizeable proportion.
So clearly energy auditors could be considered to have a serious responsibility when it comes to proving the financial and environmental benefits of energy efficiency, and to promote investments in energy efficiency.
And this is why trust is so important.
However, unfortunately, the term “greenwash” has emerged and become prominent in recent times, and not without reason. Greenwash is commonly applied in reference to companies which exaggerate their environmental credentials.
For example, the Corporate Climate Responsibility Monitor Report for 2022 assessed 25 major multinational companies on their “net zero” claims. The report found that “net-zero targets aim to reduce the analysed companies’ aggregate emissions by only 40% at most, not 100% as suggested by the term “net zero”.
It is my opinion that there is also greenwash when it comes to several energy efficiency related certifications. As I have outlined above, after passing a multiple-choice exam you haven’t really demonstrated the highest level of competencies an energy auditor needs, and the evidence of competency is relatively weak.
An individual achieving energy efficiency certification by passing a multiple-choice exam may or may not be competent as an energy auditor, but its impossible to really know with much confidence.
Labelling someone as a “professional” after passing such an exam is greenwash. The certification gives the appearance that the person is competent, but competency has not actually been sufficiently proven to be able to make this statement.
And what are the consequences of greenwash?
Cynicism. Caution. Avoidance. Distrust.
But what happens when there is trust? In his book, The Speed of Trust. Steven M. Covey identifies that “when trust goes up, costs come down and speed goes up”.
For those people providing energy efficiency advice to be trusted, we need certifications that are trustworthy. Such as the Investment Grade Energy Auditor certification from the Sustainability Education Academy.
*Note: Whilst in theory multiple choice exams should be inexpensive to grade, the costs of the various multiple choice based energy efficiency certifications is actually similar to the cost of Investment Grade Energy Auditor certification.
