Take-up & use

Take-up & use covers the period of deploying the service, recruiting users and operation.

Confirmation Timelines and Contract

Ensure that all required roles are properly covered and reflected in formal agreements or contracts, as applicable. Formal agreements should also be made about the deadlines for service provision and finalisation of activities, including:

Preparation of installation & operation

For each building (with the exception of testing), the following items must be agreed with the trades and contractors to be invited.

• Prototype testing without users
• Prototype testing with users
• Dwellings/Zones: Equipment installation (a) (e.g. meters)
• Dwellings/Zones: Equipment installation (b)(e.g. wiring)
• Buildings: Equipment installation (a) (e.g. concentrators)
• Buildings: Equipment installation (b) (e.g. wiring)
• Distribution of Consent forms
• Collection & validation of consent forms
• Selection of site for on-site testing
• On-site testing
• Feedback platform (e.g. Web portal) going online
• edss: Pilot Operation Start
• ems: Pilot Operation Start
• add any other relevant activity here ...

Preparing Operation

A “Help Desk” should be put in place to develop solution scenarios to respond to problems faced by service users during operation period. The person your user is going to call, should be know to whom it should be directed and / or what the answer is. Hence, for all sites main challenges to be faced / expected during service operation phase must be identified. An appropriate organisational environment with clear and well defined roles and responsibilities is essential, to assure an accurate and sustainable service operation throughout the pilot site.

The individual lists might be INCOMPLETE. Please add any issue you already have a process for or had to develop during the lifetime of the project.

Main Challenges

To-do: Briefly describe what key difficulties you were challenged with or expect to overcome during service operation with regard to the main stakeholders involved.

*Collect and share the issues with your partners in document together with the filled lists below.

Description of the partners’ roles

To-do: Please describe the role of each partner in the service for your pilot site and complete the necessary information in the table below. Note: If service provision across features differs, provide roles for each service in a separate table.

For each role and stakeholder, fill the following items:

• Responsibilities
• Project partner performing the role

Stakeholders to be covered are: Tenant (T) / Staff (S), Professional (P), Visitor (V), Municipality / Council, Social Housing Provider, IT & Measurement Provider, Energy Provider.

Service related scenarios: problem

To-do: Please provide a comprehensive list of common problem scenarios that might arise during service operation. Clearly describe the solution approach to the receptive scenarios - though the problem might be the same, the path for resolving it might differ depending on stakeholder (due to rights etc.)!

For each scenario describe:

• Problem Solver
• Description of solution
• Communication means

Issue initiated by Tenant (T) / Staff (S) / Professional (P)

• Users are not able to log-into the web portal
• Users have doubts about portal functionality
• Users have difficulties in understanding the provided information
• Users detect irregular or missing data in the portal
• Users detect problem with the web application
• Users notice a general energy problem (e.g. too high consumption in the system).

Issue initiated by Visitor (V)

• Visitors have questions about the SMARTSPACES service
• Visitor is not able to get the required information
• Visitor is not able to input information or share information with others through the system’s interface

Issue initiated by Municipality / Council / Social Housing Provider

• New Tenant (T) / Staff (S) is not able to use the web portal
• Organisation detects damage of sensors/ meters/ switch / connection
• Organisation individuates problems with the metering system
• Organisation detect too low/too high/none consumption data
• Staff need to access the system room in case of a problem

Issue initiated by the IT & Measurement Provider

• Provider detects technical problems with portal
• Provider detects problems with measurement equipment
• Provider detects middleware problems
• Provider lost of data during update of software
• Provider detects irregular or missing data

Issue initiated by the Energy Manager

• Energy manager detects missing data
• Energy manager detects problem with the web application
• Energy manager staff individuates problems with the metering system
• New staff employed by Energy manager is not able to use the web portal.
• Energy manager detects high energy consumption in the system
• Energy manager detects too low / too high / none consumption data in the system
• Energy manager detects building energy system problem

Test the System

For testing the system to be implemented, it is important to define the scope in advance. This includes the following factors, to be documented in a pilot test plan. This should comprise the Dimensions tested:

• Usability;
• Functionality;
• Perceived utility from system users’ viewpoint.

The test plan should also define the process of Recruitment of test persons from the target groups such as Tenant (T) / Staff (S) and Professional (P).

It is desirable to involve several members of each group in the test process either individually (individual testing) or as groups (joint testing).

Prototype set-up: Typically, the prototype technology to be tested is the visualisation of energy data to tenants, property managing and energy services companies through energy awareness system applications (such as web portals, mobile apps, in-home displays, etc) and/or energy management systems.

Test Regime – Walk-through with Use Cases: Prototype testing is carried out as walk-through the Use Cases and the related processes from the process models. Participants should be invited and encouraged to ask questions and comment at appropriate stages along the way. After completion of the demonstration/walk-through an open discussion should be encouraged where the participants have the opportunity to ask further questions to the presenters and together with the presenters elaborate on specific system features of interest.

Location: There are various options for where testing to take place. The selection of an appropriate location should take into account:

• Availability of technical infrastructure;
• Number of participants;
• Time available;
• Accessibility.

Prototype testing

For prototype testing, test protocols based on the defined use cases are drawn up. The implemented test prototype is subjected to laboratory based testing:

• firstly without users, for which purpose end-user and staff input is simulated or generated automatically as appropriate;
• secondly with users from all target groups (tenants, social housing staff etc.).

The prototype is aimed to provide system users with a detailed view of the proposed system solutions in order to provide feedback on their usability, functionality and an assessment of the value for system users. Results help to drive system implementation to ensure successful start of pilot operation.

The test plan should provide all use case diagrams and descriptions for edss and ems, as prepared during the Technical set-up phase. Besides the use cases also the requirements following from user characteristics should be tested on-site.

Feedback from the sessions needs to be collected in a structured format. For this purpose, a user questionnaire should be developed for gathering feedback from test users, containing questions on the user experience and covering all applicable use cases. Make sure to use closed questions (i.e. multiple response) wherever possible but also plan in open questions to allow for response about issues not foreseen by the developers. For every use case test persons are asked to:

• Run through the steps comprising the specific use case;
• Provide feedback on their experience using a Likert scale (e.g. from 1– very low to 5 – very high) and separately for usability, functionality and utility;
• Provide feedback on any features missing from a user perspective;
• [for system providers only] Provide feedback on integration of the new solutions with existing systems to be retained and operational/business processes;
• Add further notes in free-form text, containing remarks, suggestion, etc. (optionally).

Hint

With the interactive list of requirements from empirica you can track each requirements and their fulfilment (and relevance).

User statements and test results will then be fed back into the evaluation and (if necessary) final development process.

On-site testing

The on-site test represents the final test phase in which all the functional and non-functional requirements should be addressed in the prototype system implementations to be tested. To do so, the generic used case diagrams and descriptions are to be adapted to the implementation context.

Whereas the prototype testing phase focuses on some specific aspects of the systems behaviour, on-site testing is aimed to provide a comprehensive overview on the fulfilment of almost all the requirements defined at earlier stages of the implementation project. In order to guarantee smooth operation the testing is designed to reveal problems that might arise from the particular situation of equipment, communication networks, and the organisational environment in which the service staff work. On-site testing follows a similar methodology as prototype testing, but the applications run with data from the installed metering equipment in real pilot operation.

On-site testing should be carried out by the equipment/ICT provider in cooperation with key service provider / housing staff. A reporting template has to be developed to allow pilot managers to report on the testing of the following key issues:

• Test of the measuring equipment correctness after installation at pilot site. Incorrect measurements may result from faulty installation, equipment failure or unacceptable bias. For this a number of checks have to be performed, e.g. comparison of readings of installed meters with that of the utility over a certain period; cross-check measurements using portable network analysers, etc. Heat metering measurement correctness can be verified using cross-comparison of main and individual meters. Heat cost allocators do not directly measure heat flows and cannot be compared explicitly for correctness at the pilot site, so their accuracy should rely on approved quality procedures of the manufacturers and suppliers of these devices.
• Test the connectivity of pilot site equipment. This includes the check for user account configuration to visualise the correct data to the correct user (data mapping), the capability of the system to detect communication failures and the data storage capacity (buffer time) of the data acquisition equipment in case of communication failure. The appropriate configuration of the system is checked to avoid problems with data loss during operation.
• Test of service applications at pilot site level. The testing of the application at pilot site level should be performed by the pilot operating staff following the methodology described in the previous point, in order to serve as a final approval.
• Test of the organisational environment for service operation. The roles of each participant in the service at the pilot site are clearly outlined and their preparation for service is reported. A set of common operational problem scenarios is checked in order to test the preparation of the pilot organisation for service delivery.

The results are then summarised in order to calculate average level of satisfaction for each use case. Further improvements focus on addressing testers’ feedback and improving the implementation of the use cases that they were less satisfied with.

Recruitment

Staff Training

Well informed employees who interact with users play a key role for the success of the implementation. When in contact with consumers, staff should be able to answer standard as well as site-specific questions. There should also be a thorough understanding about the benefits of smart metering to all parties concerned, as well as a common vision revolving around responsibility for the environment and commitment to the interests of tenants – especially if these are households at risk of fuel poverty.

Type and scope of staff training can take several forms and depends on the services to be implemented. Of course, technical staff needs to be trained in any new system to be implemented. Importantly, housing company employees who are in direct, day-to-day contact with tenants need to be trained as tenants are most likely to approach them first when they experience problems or have open questions. Back-office staff will need to understand the system so that they can administer changes of customer data, for instance when tenants move out or in. The staff installing the meters should be properly trained as well so that they can answer consumers’ questions but – importantly! – also communicate the advantages of the technology rather than contribute to feelings of concern and uncertainty.

The Change Process

A key step towards a successful recruitment process is to understand how change is perceived by individuals. Studies have shown that they tend to follow a predictable pattern of reactions, known as a transition or change curve. The proposed strategy for the SmartSpaces project has been aligned to take these patterns into account. Presented here is a simplified variation of the Kubler-Ross change curve. [1]

When confronted with something new, citizens will typically experience shock and denial at the beginning. They will avoid the topic, refuse to believe a change is happening, and show no initiative because they are happy with the status quo. The main reasons for this are lack of information, fear of the unknown as well as feeling threatened by the new technology and experiencing fear of failure. In a next step, they will be resistant as to the adoption of the new idea, show no accountability and will generally try to find a scapegoat to justify their non-committing to the change. With the proper tools, citizens will slowly come out of this state and begin to explore the new solution. Active support and training is a must in this phase. When done properly and consistently, this will lead to the citizens committing to the change and ultimately will help to reduce the energy consumption in the building and save municipality money. How the different stages are addressed with the approach is presented in more detail in the next section.

—— Footnotes

[1]	Gilley, Jerry W., Scott A. Quatro, Erik Hoekstra, Doug D. Whittle, and Ann Maycunich. The manager as change agent: a practical guide to developing high-performance people and organizations. USA: Perseus Publishing, 2001.

Recruitment Strategy

Note

This section is based on a paper published by Strahil Birov and Georg Vogt the eceee‘15 summer study. More detail and strategies for all sites can be found in the public document D5.1 available on the SMARTSPACES website.

The strategy developed within the :ref:`SMARTSPACES-content project is a result of multiple meetings and project workshops with professional and technical staff from all eleven pilot sites in eight countries:

• inclusion: engage all citizens – of different age, occupation and level of technology awareness;
• balance: find the right mix and number of recruitment measures, as too many measures may overwhelm the audience, too few can result in low level of interest and participation;
• coherence: eleven pilots in different parts of Europe were required to apply the strategy and successfully recruit the target number based on the size of the pilots. The different pilots assessed possible risks and planned how to manage them to ensure successful application of the strategy.

Core strategy elements presented below are aligned with the way users perceive the topic of energy saving through the use of ICT and the need to commit to reducing CO2 emissions:

1. In order to overcome the denial they may have, they are provided with catchy messages and support materials as part of a long-term branding plan.
2. For internal users (regulars in the buildings), their resistance is effectively overcome by leveraging the structure of the organisation.
3. In order to explore the new possibilities energy-saving measures offer, users need to be trained and provided with more details in order to get their buy-in.
4. Committing to the change of using ICT to save energy in buildings is solidified by the continuous branding, assistance and also by introducing the notion of champions.

As-is analysis – get to know your audience

The best suited strategy for each pilot site depends on the results of the pilot’s as-is analysis that delivers information on the profile of staff/visitors in the buildings, the hierarchical strength, the organisational relationships (e.g. boss – colleague or city mayor - executive), goals, values, etc. In the case of SMARTSPACES, the as-is analysis was performed by means of qualitative research. It involved identifying the target audiences in the buildings (age, interests) and their needs, as well as available channels for communication, suitable materials and incentives, and possible problems that may occur during recruitment.

For any stakeholder wishing to use the strategy, the as-is analysis should include at least the following:

1. Audience profile: Who are the target audiences, what do they like, which communication channels do they prefer?
2. Organisation: Can organisational relationships influence the changes, who are the change initiators?
3. Materials: What are suitable materials and incentives for the different audiences?

Message development – catching attention

Messages are used to communicate with and encourage individuals to change their environmental behaviours [2] . Such were developed in accordance with the pilot-specific situation in SMARTSPACES. Helping questions include ‘Is the message involving, important, relevant to the receiver?’, ‘Is the information useful for busy people?’, ‘Is it professionally / convincingly delivered?’, ‘Is the source credible / trustworthy?’, etc. All messages should tick the boxes of being relevant, interesting, credible, clear, convincing, and come from a trusted source. Macnamara’s (1992) [3] pyramid model served as a guiding source for the message development process, as it gives a systematic overview of media evaluation divided into three stages – inputs, outputs and outcomes. The complete process of creating such messages identified by the consortium involves the steps described below.

• As-is-analysis
• Develop initial message concepts
• Assess message concepts
• Fine tune messages
• Validate messages

Messages should be oriented around topics that are close to the target audience. For example, in an economic crisis with job uncertainty, municipal workers would most likely support the idea that energy saving contributes to saving municipal money and therefore helps to keep jobs. This was observed in pilots like Bristol and Murcia, but apart from financial topics others include environmental and ecological, cultural and institutional influences. The messages are delivered through different channels. The media richness theory (Heeren and Lewis 1997) [4] alludes to the importance of having the right mix of channels in order to avoid oversimplified or overcomplicated communication. Channels like face-to-face and video meetings offer the most effective results and should be applied when possible. However, the right mix should be selected based on the as-is analysis for each individual case.

Materials and incentives – fighting the fear of the unknown

If various sites are involved, a centralised database should be implemented to share and exchange ideas, reuse of materials and general knowledge collection. Sharing templates enables quick adaptation and reduces costs. All materials and activities should be tailored to the needs and expectations of the different audiences.

Various materials and incentives have been developed for this purpose, including:

• leaflets and brochures
• posters and banners
• articles and other publications
• newsletters
• video materials
• social media
• presentations and workshops
• games and quizzes
• free services/bonus system
• energy coaching services

Branding – providing consistency

The way in which an idea is presented influences dramatically the idea’s success. For example, Lewis et al. (2012) [5] explain that the initial message proclaiming initiatives to save energy and money should come from communication sources for which the consumer associates no negative vested interest; a source that they feel is independent, ideally on their side and/or the side of the environment, but at least not on the side of the utility companies. These and other considerations suggest strongly that the recruitment strategy should have a clear view on what the face of the initiative/project is.Typical related questions include:

• ‘Who is providing the services – a private company, the municipality, etc.?’,
• ‘How are the services and/or products branded – under SMARTSPACES, under the municipality programme, under national initiatives that have more authority and respect among people, etc.?’,
• ‘Who has already won people’s trust and can be used to endorse the SMARTSPACES services?’.

The organisational structure – overcoming resistance

As public buildings are owned or controlled by the state (municipalities, city authorities) and most of them have an established structure (most commonly mechanical and therefore not flexible but strongly hierarchical), especially true for administrative buildings, one should take into consideration the value the structure could provide when it comes to communicating an idea and inducing change in an organisation. There are different approaches for initiating such changes each of which possesses advantages and disadvantages. [6]

Utilising the influence of the structure is very helpful for overcoming the resistance of the internal users when it comes to change. The most common models in the case of SmartSpaces suggest the use of a top-down-strategy where the top management’s authority is undisputed and actively involved (city mayor, department executives), a bottom-up strategy where the users bring forth a request for change that is acknowledged by the top management, or a mixed strategy combining the previous two. The best suited strategy for each pilot site depends on the results of the pilot’s as-is analysis that delivers information on the profile hierarchical strength and the organisational relationships (e.g. boss – colleague or city mayor - executive).

Training – facilitating exploration

Trained and motivated users have the ability to further disseminate among their fellow colleagues or among visitors, creating a cascading effect that has the potential to achieve high user awareness and education. Effective training includes multiple sessions or focus group with support materials such as manuals and instructions. Designated contact people for each building combined with a hotline are further facilitators for supporting users in their exploration of the new technology.

The concept of champions was utilised in SMARTSPACES to expedite the wider update. Champions are individuals who serve as role models to others and influence them through their actions and opinions. A champion can be anyone with good social skills. In SMARTSPACES they are seen as ‘social hubs’ that have great potential in raising awareness and recruiting new users. Therefore, the process of identifying these persons in each building has begun early on in the project. Identification instruments include the use of a survey to screen for potential champions, observation during focus groups and audits, and the use of contests where specific characteristics (champion profiles) are sought.

—— Footnotes

[2]	Wilson, Caroline. “Evaluating communication to optimise consumer-directed energy efficiency interventions.” Energy Policy, 19 August 2014.

[3]	Macnamara, J. “Evaluation: The Achilles Heel of the Public Relations Profession.” IPRA Review, 1992.

[4]	Heeren, E., and R. Lewis. “Selecting communication media for distributed communities.” Journal of Computer Assisted Learning, 1997.

[5]	Lewis, Ph., Chr. Dromacque, S. Brennan, J. Stromback, and D. Kennedy. “Empower Demand 2: Energy Efficiency through Information and Communication Technology - Best Practice Examples and Guidance.” 2012.

[6]	Vahs, Dietmar. Organisation: Ein Lehr- und Managementbuch. 8. Edition. Stuttgart: Schaeffer-Poeschel, 2012.

Lessons learnt

Buy-in of senior management

• Impact

  • Senior manager can also communicate on higher level to other departments affected
  • It is especially difficult to build relationships during organisational change
  • Without senior support, conflicts can arise and communication efforts needed might increase

• Recommendation

  • Find a strong project sponsor in a senior position that can bring together different service areas from the outset i.e. project inception
  • Focuse initial communication campaign rather than trying to reach all levels of the hierarchy instantly

Ensure maintenance |ss| and |sp| are aware of the project

• Impact

  • If not aware, they might consider service as a tool to replace them
  • They can be core users with workable knowledge of buildings

• Recommendation

  • Explain how savings make sure that budget cuts will not be necessary
  • Offer training and reduce fear of not being able to handle ICT
  • Make, occasionaly, service an agenda item at meetings in department

Training and information sessions need to be tailored to the audience

• Impact

  • Telling regular users too much, will make the unsure about what to do and to reduce risk, not use the service
  • Telling professionals what they already know demotivates them and the session is perceived as waste of time

• Recommendation

  • Find out what the current level of knowledge is and focus on new learning
  • Keep it short (better 2 x 1hr sessions than 1 x 4 hrs) to digest the contents
  • Be clear about what it is that we want to achieve and why we need their help

Workshop size and organisation

• Impact

  • Group training enables users to exchange also after session
  • Contact

• Recommendation

  • For Professional: small groups of 6-7 (ideally a working team)
  • For regular users: mid-sized groups of 12-17 (ideally neigbhours)
  • Provide small gifts (e.g. mugs) with link as reminder about service

Authority sometimes required during implementation

• Impact

  • Some Staff consider energy to be their domain alone and react defensive
  • Presence of department head in meetings eases tension and avoids unnecessary conflict escalation

• Recommendation

  • Indentify early on which communication strategy is needed (top-down?)
  • Involve, if applicable, the department of occupational risk prevention

Establish a communication strategey

• Impact

  • Terms should be recognised in context of service
  • To frequent messages will appear random and confused

• Recommendation

  • Define terms and communicate to Professional, champions, help desk etc. to use when talking about service
  • Define a few messages and present them in certain context, e.g. spot on the internal website
  • Select a few channels to communicate to focus your effort and reduce cost

Have a trusted organisation as a partner in public meetings

• Impact

  • Institutions such as universities enjoy the confidence of the employees in the public sector and can achieve the advantage in your favour

• Recommendation

  • Invite and inform universities, associations, unions etc.
  • Consider a regional university over “consultants” from outside

Identify champions, ideally in each building

• Impact

  • Champions will take responsibility and contact the professional when they cannot achieve result themselves
  • edss identifies waste and opportunities, but action is required by individuals

• Recommendation

  • Identify energy champions in workshop and recruitment events
  • Establish link to dedicated energy and other building professionals to carry out specialised works

Setup information desk / kiosk

• Impact

  • Especially a Visitor waiting will check out of interest
  • Content is mostly already available on portal, just needs to be linked or re-used
  • Makes service part of the “real” and not “only” digital world

• Recommendation

  • Create simple design ideally using logo of service
  • Setup kiosk, visibly, in waiting areas at location where Visitor can stand for a few minutes without blocking the way
  • Kiosk should be rebootable to avoid maintenance effort
  • Use to communicate success achieved including monetary savings