Future Melbourne Wiki: From Consultation to Participation

A paper produced by Dr. Mark Alan Elliott of Collabforge (www.collabforge.com) for the City of Melbourne's Future Melbourne project, 2008.

1. Executive Summary

This document outlines an important evolution in the process and outcomes of the consultation process, through the use of wiki technology. By editing a shared document (the City plan) the creativity of the public can be engaged through leveraging collaboration, this having the added benefits of fostering a sense of shared ownership and legitimacy in the project. In addition, this platform provides an opportunity for interactive, transparent and open engagement which bridges the municipality-public line like never before, enabling the possibility for mass collaboration across the City's community and beyond.

As this project represents a world first in the application of wiki technology at this scale - across both the City of Melbourne / stakeholder collaboration and public consultation - the methods for monitoring and evaluating engagement requires redevelopment in order to suit this new paradigm. Future Melbourne will be setting a benchmark precedent in this regard as other wiki initiatives (largely pilot projects) have yet to develop a comprehensive approach to evaluation created specifically with the complexities of collaboration in mind. A broad-based and comprehensive approach will be employed which draws upon both quantitative & qualitative evaluation, as a human eye is often needed to determine the relevance and quality of participation in this context. An abundance of statistical data regarding engagement with the website will be generated and synthesised in order to provide a strong quantitative basis for engagement evaluation, while innovative techniques such as collaborative 'wikerviews' (interviews conducted within the wiki) will be used to leverage the platform's capacity for gathering qualitative data as well.

The collaborative engagement and interaction of the public, officers and stakeholders presents a groundbreaking opportunity for explorations into the evolution of democracy towards participatory governance. Not only does the wiki have the power to "improve transparency, efficiency and effectiveness of regulation" - the purpose of public consultation according to the OECD,¹ but it also has the capacity to engage the public in a far more inclusive, interactive and meaningful way through opening up for collective redrafting the actual policy under review.

2. Why Wiki?

2.1. From Cooperative to Collaborative Participation

Utilising wikis for public consultation entails a fundamental shift in process. This shift can be best described as moving from cooperative to collaborative participation, where 'cooperative' indicates an outcome reliant upon individualised, linear procedures, while 'collaborative' signals a collective, nonlinear creative process. ² Cooperative processes employed in traditional consultation typically define how the public can engage by specifying a narrow window of procedures, e.g. compose and submit a comment in 'x' media which may then receive a similarly formed response. The application of cooperative procedures in such cases result in an output of individual, commentary oriented interactions.

Wiki consultation varies from this process in that while the public may still contribute comments, they are also asked to edit the document up for evaluation. This process is inherently collaborative as other participants also edit the same document, introducing 'nonlinear contribution' (edits may occur simultaneously anywhere within the document and they may effect each other). The result is a document that evolves throughout the consultation period with specific contributions being subsumed (incorporated, modified, moved, deleted, developed etc.) in the process. This changes the nature of the outcome in that instead of an output of discrete interactions between individual members of the public and municipality, the result is an integrated whole, where interactions are indirect (mediated by the document), dispersed throughout the work and may occur between anyone participating.

2.2. The Wiki Advantage - Enabling Mass Collaboration

The wiki, a website that allows users to edit and save content, constitutes a shared workspace which mediates the collaborative process. This mediation is the critical advance in collaborative technologies as it provides a degree of separation between the participants involved, enabling direct contribution to the creative task at hand without the necessity for social negotiation (imagine the impossibilities of having to become acquainted with and maintain relationships with thousands of people for a specific project). Social mediation via a site of work accessed through digital networks (allowing multiple people to edit the same document at the same time) enables 'mass collaboration' - the scaling of membership beyond the marginal limits of approximately 25 collaborative participants³ towards that of hundreds of thousands and beyond.⁴

In addition to the benefits of social mediation, wikis provide for the modularisation of work - the work can be broken down into many various types of tasks - e.g. fixing typos, rewording, original ideation, standardising, reformatting etc. This modularisation provides for the accommodation of a wide range of interests, orientations, capacities, skills and perspectives - integrating a broadly inclusive creative process into its very design. The wiki and other mass collaborative technologies⁵ have provided humanity for the first time with the ability to break through the glass ceiling of collaborative membership. They also enable the capacity to implement the collaborative process in order to leverage benefits associated with its process. Such benefits include generating shared ownership and understandings, consensus and collective intelligence across entire communities.

2.3. Effects, Benefits & Challenges of Collaborative Consultation

2.3.1. Effects

Employing collaboration as part of the public consultation process means moving from limited, linear and systematic procedures for specifically defining engagement (e.g. the public meeting in a Town Hall or writing a letter to Council) to a process which openly accommodates and encourages creative engagement and as a result is extensive, nonlinear and systemic in nature. This changes the way contributions are considered and responded to in important ways. Instead of yielding discrete comments to be considered and synthesised by officers and councilors, the contributions are addressed by the collective audience of participants in real-time. The merits of each submission are then reviewed by participants and democratically included, modified or omitted - often accompanying discussion. Therefore, editing the plan can be thought of as a form of 'integrated commenting' which provides the capacity for participants to demonstrate their ideas in context. Not only is this an entirely new and different mode of participation from previous consultation methods, it also provides the opportunity for participants to,

• engage creatively with the actual document up for review,
• engage and build relationships with all stakeholders involved,
• develop a stake in the outcome, thereby increasing voluntary compliance and commitment to the plan through a sense of increased legitimacy and shared ownership.

2.3.2. Benefits

The development of shared ownership is an obvious benefit of collaboration, however one of the more important opportunities presented in the use of wikis in collaborative consultation is providing multiple pathways for engagement. Participants may read and comment on the plan via the use of discussion pages, as well as contribute edits to the plan directly. This means the participant will self-select tasks within these activities along a great variety of lines according to the modularisation discussed above - some will choose to discuss while others will focus on editing the plan, while still others might involve themselves in moderating discussions and editing. This self-selecting dynamic adds an important benefit to the process as participants can influence,

• the context in which their input is interpreted,
• how their input will be used, and
• what forms of responses they are likely to get (e.g. a comment from an officer or a development/refinement of an idea within the plan).

Another benefit provided from the specific wiki structure employed in this project is the use of discussion pages (one for each content page) where participant commentary can be evaluated and addressed as an integrated part of the consultation process by officers, council, stakeholders and other members of the public.⁶ The iterative nature of an unfolding discussion (as opposed to discretely submitted comments) provides for participant satisfaction to be assessed and addressed as a natural part of the wiki process. In addition, as conversations are associated with corresponding sections of the plan that are undergoing editing in parallel, both evolve in concert with one another, providing a considerably more interactive, integrated, contextualised and comprehensive review process.

2.3.3. Challenges

While wiki-based collaborative consultation is not without precedent (see below) it is new for most councils and their constituents. Therefore, it is important to effectively communicate its opportunities and what participants can expect in regard to how their contributions are to be considered, addressed and or included (e.g. 'Due to the collaborative nature of contributing to a wiki, your edits will become an integrated part of the plan's evolution. As a consequence your edits may be incorporated, deleted, transformed, moved etc. according to the consensus of the collaborative community.') Another challenge presented is developing an approach to evaluating engagement appropriate for this new process.

Parameters for consultation evaluation are traditionally determined by tracking and measuring the number of public and officer interactions (e.g. comments and responses). This is effective specifically because interactions in cooperative contexts consist of linear, discrete and discontinuous elements. Collaborative contexts are considerably more complex in regard to the interactions that take place and the diversity of inputs which contribute. Nevertheless, quantitative evaluation of engagement is possible and a very detailed picture of the activity can be developed through providing a wide range of statistics. In addition, qualitative evaluation will provide added insight regarding elements of engagement not so readily captured by quantitative statistics, e.g. tone, character and orientation of interactions as well as perceived levels of satisfaction regarding addressing issues raised by participants.

The following section provides a detailed strategy for monitoring and evaluating the engagement of collaborative participation.

3. Collaborative Engagement Monitoring & Evaluation

3.1. Future Melbourne to Set a Benchmark Precedent

While other projects have utilised wikis for public consultation, none so far have been found to be utilising the technology in its full potential in the way that the Future Melbourne project is. For example, the New Zealand Police Act Review utilised a wiki as a means to collect ideas and responses from the public regarding a review to the nation's Police Act. While the story was picked up in the international press which resulted in hundreds of contributions from around the world, the wiki was used to gather alternative views and commentary inline with the Act's content.⁷ In other words, the wiki provided a scratch pad of sorts upon which the public could provide alternate wording/views as well as commentary. Future Melbourne differs considerably in this regard in that there are separate discussion pages for commenting on the various sections of the plan. This is because the public can make changes to the actual plan itself and thus commentary must be kept separate.⁸ Further, only cursory attempts were made to evaluate the Police Act wiki, as is was not a core component of the drafting or consultation processes. In fact, the wiki component of the Police Act review was only considered complimentary to their traditional modes of cooperative consultation and was primarily run as a trial of the technology.

Overall, the wiki Act produced several hundred constructive edits ranging from single word suggestions through to lengthy paragraphs of commentary. The wiki Act has yielded at least three positive outcomes:

• Firstly, a good number of fresh ideas and the development of existing ideas.
• Secondly, awareness and engagement of people in the review of this legislation was significantly increased.
• Thirdly, while not a direct objective of the wiki Act, its release has nonetheless developed a deeper conversation on government use of new technologies and online social networking spaces (both within government circles and in the wider cyber-community).⁹

Another project to make use of a wiki for consultation is the Afton Comprehensive Plan wiki (Minnesota, USA). While it appears that the town's plan is being developed in the wiki, there is far less scope in regard to public participation,

Anyone can view the comprehensive plan online, but only Afton residents who register will be able to make comments... People who want to rewrite or edit the plan must ask for permission to do so and will then be asked to attend the city's comprehensive plan advisory committee meetings, Marohn said.¹⁰

In other words, at this point the wiki is not being used for collaborative consultation, but rather traditional cooperative engagement via standard online commenting. At this point there is no visible indication of an evaluation strategy.

There are several other such wiki projects whose focus on measuring engagement is limited to the number of site visits and edits. This is an approach oriented towards the discrete/discontinuous process of cooperative contributions, but is ultimately inadequate for accurately measuring the complexities of collaborative engagement for reasons outlined below. Therefore, Future Melbourne can be seen as a world leader in not only the comprehensive application of a wiki to the city planning process, but also in regard to the development of a participation monitoring and evaluation framework that was conceived and designed specifically for the demands of wiki-based, collaborative consultation.

3.2. An Expanded Notion Of Participation

Considering that a great many inputs are involved in the collaborative process, a broad based approach is required for measuring engagement. This means incorporating some elements that were previously outside of the evaluation gambit and consequently expanding the traditional view of 'participation'.¹¹

While insufficient on their own, various types of page views become a relevant and important component of collaborative engagement - i.e. how much time do participants spend informing themselves by reading on the site in addition to making contributions?¹² It is also important to consider page views by visitors that are not logged in as this is another important subset of participation, albeit one less engaged. Similarly, the level of awareness the wider community has regarding the project will also effect engagement rates and so tracking site referrals from online press becomes an important, broad stroke indicator of community participation.

Since wiki editing tends to be highly modularised with levels and types of engagement appealing differently to different participants at different times, it can be extremely challenging to determine levels of engagement through statistical analysis of site usage. However one of the most telling and important measurements of public engagement (especially in complex, collaborative contexts) are the levels of satisfaction achieved by individuals with regard to their perspective of their own engagement experience. For instance, a participant might learn a great deal from following the evolution of the plan but without ever contributing substantial edits. While they may have only contributed a minor element (for instance fixing a typo), their level of engagement might have actually been much deeper through an educational experience tied to following the evolution of the concepts at stake.

3.3. Quantitative Evaluation

Measuring this level of 'engagement satisfaction' will be done through holding an online poll which records (as well as aggregates) individual participant feedback. This poll will host a variety of multiple choice questions as well has have the capacity for free form entries, thereby providing fine-grain statistical data regarding the participants' personal experience of engagement satisfaction.

While measuring engagement satisfaction provides an excellent portal into the individual participant's experience, a great deal can be learned from broad-stroke quantitative evaluation. The wiki's native revision tracking system in combination with site and server's logs provide an abundance of statistical data regarding engagement with the website. In addition to the above quantitative measurements, a wide range of data sets can be used to measure:

• the date and time of registrations
• demographics
• how many times a user logged on
• traffic flows of registered and unregistered users
• the specific pathways users took through the site and how long they spent on any given page
• how many saves they made in any given period, or overall
• breakdowns of saves per users or by groups
• what were the most popular topics and how many views they received
• (other sets of data and their intersections are also being explored) For examples of such statistics, see the attached 'Public Consultation 1 Community Monitoring Report'.

While representations of such data will provide an in-depth perspective of engagement with the site, it is still challenging to represent the full detail of collaborative engagement due to the complexity of interaction. For instance, viewing how many saves were made provides no indication of contribution quality, not to mention that a user may save many times during one editing session in order to backup their work. While determining matters of quality and specific quantity of contribution will largely be a role for qualitative evaluation (see below), there are tools already available to greatly assist in the quantitative evaluation of contributions.

The system's capacity to compare the differences between revisions and provide a side-by-side view enables a quick and efficient means of evaluating the evolution of any given page, and as a consequence, how contributions from various participants may have evolved.

Figure 1
Two revision differences, newer revision on left, older on right.

3.4. Qualitative Evaluation

Qualitative evaluation is critical in collaborative contexts as many of collaborative participation's complexities are beyond the reach of quantitative analysis. As discussed above, a human eye is often needed to determine the relevance, tone, quality and context of a participant's contribution. The majority of this evaluation will be conducted during the consultation period with project course-corrections and participant interaction adjusted in real-time in order to foster ever higher quality of contributions. This real-time monitoring and evaluation of interactions will also help ensure that conflict resolution mechanisms are triggered should a debate tend toward argument. Such incidents left unchecked can devolve the activity in the direction of vested interests, biased perspectives and one-upmanship.

Utilising the online platform, surveys and polls will be conducted with a range of individuals across all participant groups, in order to gather feedback that might otherwise fall through the cracks. A number of participants representing different styles and subsets of participation will also be invited to be 'wikerviewed' - an collaborative interview conducted within the wiki. The wiki medium enables multi-pathway, collaborative interviewing, where multiple people can participate in an interview with multiple interviewers. In addition to garnering in-depth information regarding engagement levels and experiences, this will provide an additional forum presenting opportunities to discover and address any gaps in participant concerns and interests.

3.5. Holistic Evaluation

Upon completion of the consultation period, the wiki will be heavily encoded with the ideas and opinions expressed by those who edited, while page views, saves and registrations will provide a strong indicator of the proportion who utilised the wiki's functionality and in what ways. Analysis of this 'whole' created by the collaborative community will in the end provide the most telling indicators of the levels and success of engagement. A version of the site will be archived immediately upon completion of public consultation, before the refactoring of content which will need to take place prior to submission of the Plan to Council. This archival record will serve as a reference point for ongoing analysis on the part of the Future Melbourne team, and should be considered being made available to the public as an easy point of comparison with the plan adopted by Council.¹³ An archival version will also provide an excellent learning resource for other municipalities interested in collaborative consultation.

3.6. Monitoring, Evaluation & Reporting Summary

In the context of collaborative consultation, the traditional methods of tracking and reporting individual comments, on the whole, loses its relevance. Since collaborative interactions are far more iterative, nuanced and responsive than cooperative interactions, in some respects, the evaluation of engagement becomes less about reporting who said what when and more about engaging the community in real-time during the consultation period. Therefore, an approach must be adopted which seeks to integrate and represent a wide variety of engagement vectors in order to build a detailed picture from the diverse range of inputs required. To draw upon an analogy (and mix it with a metaphor), with the advancement of the colour TV comes the ability to more accurately represent reality, however one also loses the ability to make 'black and white' judgments about what detail the representation is comprised of. Similarly, collaborative consultation provides for a quantum leap in the advancement of participation and engagement 'resolution', however as a result, it is also likely to blur some edges along the way.

4. Conclusion: Towards Participatory Governance

The shift from cooperative to collaborative consultation signals a critical point in the evolution of democracy.¹⁴ Opening up matters of policy for the public to not only comment on, but to contribute to directly is tantamount with participatory governance. While representational democracy is unlikely to be transformed overnight, projects such as the Future Melbourne wiki are contributing to a slow but steady evolution, as the adoption of innovation having the power to "improve transparency, efficiency and effectiveness of regulation"¹⁵ can only be a matter of time. The wiki is one such technology with its rapid uptake in a wide range of high profile NGOs, corporate and public agencies underlining this fact. The capacity it brings for collaborative involvement in the governance process makes it an ideal tool in promoting the active engagement of the community, and in this regard, its functionality is almost ideal according to the Victorian Local Governance Association's definition of engagement:

Engagement is achieved when the community is and feels part of the overall governance of that community. It is informed, connected and feels it has a role to play.¹⁶

The use of collaborative wiki technology for drafting the Future Melbourne Plan lays the essential groundwork necessary to achieve widespread community engagement in the process of determining Melbourne's future. Through collaborating on this future, Melbourne's community will also be contributing to a present reality of participatory governance.

References

1 : Background Document on Public Consultation, OECD, (Code 10-Mar-2006). Paper prepared by Delia Rodrigo, Administrator, and Pedro Andrés Amo, Consultant, Regulatory Policy Division, Public Governance and Territorial Development Directorate.

2 : Elliott, Dr Mark Alan (2007) Stigmergic Collaboration: A Theoretical Framework for Mass Collaboration. PhD thesis, Centre for Ideas, Victorian College of the Arts, University of Melbourne. See http://mark-elliott.net/blog/?page_id=24

3 : Lipnack, J., & Stamps, J. (2000). Virtual Teams. Canada: John Wiley & Sons.

4 : Wikipedia, the World's largest single mass collaboration has 6,838,923 registered participants as of April 8, 2008. This number does not include unregistered editors.

5 : For a mass collaborations in various media, see a) drawing: Drawball.com; b) virtual worlds: SecondLife.com; c) computer software: Apache.org

6 : This functionality was inspired by Mediawiki, the wiki software used and developed by the Wikipedia community. (The Future Melbourne wiki was built using TWiki.) Most other wiki projects simply create discussion pages as needed, or embed discussion within the content to be refactored/deleted later.

7 : For an example see http://wiki.policeact.govt.nz/pmwiki.php/PolicingAct2008/ChildCriminals

8 : Note that the 'official' version of the Future Melbourne plan can easily be reverted to post-consultation as all revisions of every page are stored in the system.

9 : See Police_Act_Wiki_presentation_doc_2008.pdf pp. 4-5.

10 : See http://www.twincities.com/education/ci_8710533

11 : However it is worth noting that the Victorian Local Governance Association's Consultation and Engagement Resource Website already employs a suitably broad definition of 'participation' - "Participation means that the community is involved in governance activities." See http://www.vlgaconsultation.org.au/definitions.shtml

12 : Harvard Professor of Law, Yochai Benkler makes the distinction in his acclaimed work, The Wealth of Networks, (2006) that the quality of one's informational outputs have a direct and proportional connection to the quality of our informational outputs.

13 : However all revisions of very page will always be available in the current version of the Plan on the wiki platform, even after adoption by Council.

14 : Tapscott, D., & Williams, A. (2006). Wikinomics: How Mass Collaboration Changes Everything. NY: Portfolio.

15 : This is the purpose of public consultation according to the Background Document on Public Consultation, OECD, (Code 10-Mar-2006). Paper prepared by Delia Rodrigo, Administrator, and Pedro Andrés Amo, Consultant, Regulatory Policy Division, Public Governance and Territorial Development Directorate.

16 : See the Victorian Local Governance Association's Consultation and Engagement Resource Website - http://www.vlgaconsultation.org.au/definitions.shtml