2.1 Hackathons

Hackathons are a relatively new operating method since the term originated from the open source software developers in 1999 [⁴]. Hackathons lack a strict and uniform definition, making the event format rather vague; while hackathons in general are considered technology-driven events, some non-programming events have also been called hackathons [¹⁹] since they refer to hackathons as an event of conducting open innovation. In any case, a hackathon is a method where an organization utilizes external resources in addition to internal innovation [⁵]. This usage of external resources or a customer-driven innovation is also supported by Desouza et al. [⁶], as they describe how the industry has moved from the customer-centric innovation to the customer-focused and further, customer-driven, innovation practices. Hackathons can be seen as a straightforward method to implement the customer-driven innovation and acquire external feedback on the business practices.

The format and setting of the hackathon can vary based on the objectives and the requirements of the event [⁴]. Hackathons have been used in multiple contexts, such as educational hackathons [²²], civic hackathons using open data and public resources [¹⁸], culture hacks [³], and industrial hackathons, where the hosting organization is a company [²⁴]. In addition, large hackathon events can be organized by a specialized entity, where the challenges are introduced by the sponsors and other participating organizations. Such organizers are for example Ultra Hack (Site 1) and Junction (Site 2). The attendance can either be open to anyone who wants to participate, or the organizers select the participants based on their applications. These events, called public hackathons in this research, can combine other forms of hackathons into one event, since the participating organizations have their own objectives.

Briscoe and Mulligan [⁴] have categorized the different hackathons into two main subgroups: tech-centric and focus-centric. The tech-centric hackathons are events, which concentrate on the software development by applying specific technologies and tools. They are further divided into three groups: Single-Application (focus to improve one application), Application-Type (focus on specific platform or genre), and Technology-Specific (focus on one technology). The focus-centric events are solving complex problems, such as social issues, or business objectives, and are further divided into three groups: Socially-Oriented (social concerns), Demographic-Specific (for a specific group of programmers), and Company-Internal (internal staff).

As for the benefits and challenges of hackathons, a previous study by Komssi et al. [¹⁹] focused on hackathon events in F-Secure, a software company specializing in cybersecurity. Their study collected data from five hackathons and presented findings to describe the benefits and the challenges of hackathons. They outline the benefits, such as new products and new features to the existing products, radical ideas that would not be normally feasible in the scope of the company, and internal and external communication between the developers and other fields of expertise, as well as external collaboration. The detected challenges were the challenge of communication between fields, the continuance process with the prototype development post-event, and the intellectual property rights. They finish their research with the hackathon paradox: during and immediately after the event the audiences, the developers, and the other stakeholders have been satisfied with the event and the outcomes, but the prototypes are rarely being taken forward and commercialized, as if the hackathons are lacking as an innovation method.

As the very definition of a hackathon varies, the benefits and challenges of a hackathon are not that well understood. While the research done by Komssi et al. [¹⁹] outlines benefits and challenges, their case hackathons concentrate mostly on internal development in a company, which are focus-centric events by the classification of Briscoe and Mulligan [⁴]. Even the categorization of hackathons is difficult since while a hackathon event may be focus-centric, it is possible to have tech-centric tasks within the hackathon. In this research, the given tasks in each hackathon are classified based on the work of Briscoe and Mulligan [⁴], but the hackathons themselves are categorized based on their context.

2.2 Open Data

Open data is data, which is published for everyone to use, for any reason. Because of this, it is a powerful resource for innovation, since the availability of data invites innovation and combination of datasets [²⁰] and freely accessible data, making it a powerful business resource [⁹]. However, even though there is interest towards open data [¹¹], the value is somewhat complex to attain.

While the value generation from the open data can be seen as a complex problem [³²], research shows that the value is often indirect [³⁰] and a definite return of investment is difficult to calculate [17]. In their research Janssen et al. [¹⁷] found multiple economic benefits of open data, such as economic growth, stimulation to innovation, development of new products and services, crowdsourcing, to name a few. Similarly, Lindman et al. [²⁰] outline benefits, such as cost savings from crowdsourcing, increasing transparency, and new service innovations.

Because the return of investment is difficult to estimate, companies are not interested in opening their data. In the research by Immonen et al. [¹⁵], companies were interviewed and they were asked to publish their data as an open data source. The results were that the companies wanted to get financial benefits from their data, because of the costs of data collection. While there are business models for open data, companies require assurances that they will benefit from their data [¹²]. From the economical point of view, the biggest issue of open data is the fact, that it is difficult to make money with [³⁰], which is why it is generally avoided [¹²].

But when data is opened by privately owned companies, there are benefits but also drawbacks. An extensive literature review into the benefits and drawbacks of opening data from the private sector was published as prior work [¹³]. Based on this work, a list of six benefits and four drawbacks were found in the open data activities by private organizations (Table 1).

Table 1: Benefits and drawbacks of opening data to private organizations 

In this work, the objective is to assess these benefits and drawbacks in the hackathon events to understand and assess, why hackathons are applied more in a current business environment that opening data, while in a general sense the goals are similar. The research is focused on hackathons and the findings are then compared to the benefits and challenges found in the systematic literature review.

3.1 Data Collection

The data for this research was qualitative data, collected through interviews of the representatives of the hackathon organizers. The data was collected from organizations, who had hosted a hackathon in 2016 or 2017 and working with a consultation company, we were able to interview these organizations timely, soon after the event. Eight of the interviewed organizations had hosted their own hackathon, using external teams as participants and had used hackathon consultants to set up the event. Each organization had a representative, or a team, organizing the company-end of the event. Depending on a case, either the key representative or personnel from the team was chosen for the interview session. Before the interview, it was made sure that the interviewee had sufficient knowledge about the event. If not, the interview was rescheduled with a more appropriate interviewee.

We did not want to limit this research to only highly controlled industrial cases. Because of this, we added two organizations who participated in a public hackathon by offering challenges to multiple hacking teams through a large hackathon event with multiple organizations offering their challenges. It should be noted, that public hackathon does not require a public organization, privately owned companies can also offer challenges through these events. They were used as control cases in order to determine, if the found benefits and challenges would be only relevant for the industrial hackathons, or if these effects could be reached through other venues. Similarly, the interviewees were selected from the team of organizing personnel within the organization.

As mentioned, out of the ten hackathons, eight of them are categorized as industrial hackathons, where the organization was the direct host and in control of the participating teams, to a certain extent. Two of the events are categorized as public hackathons, where the organization offered challenges to a larger event with less control over the teams. The number of offered challenges, or tasks, ranged from one to four per event (Table 2). In some cases, such as Case C or Case F, the organization had hosted two separate events with similar, but separate themes before the interviews. The agenda for the hackathons varied, some were interested in an abstract concerning the business domain, while others offered a specialized technology and datasets to develop practical constructs. The challenges, that were offered, are categorized by the groupings of Briscoe and Mulligan [⁴] described in the previous section. In order to protect the anonymity of the participating organizations, the challenges are not described further because of the publicity and findability of the events.

Table 2: Participating organizations 

The interviews were constructed based on three major themes; the motivations and the goals of the event, the realization of the goals throughout the event, and the actions after the event. As a final topic, the interviewees discussed the whole process, their views on the benefits and challenges of this approach, and the future aspects, such as improvement proposals for the future hackathons.

The interviews were held as semi-structured: the interviewer had a list of questions prepared based on the themes of the interview, but it was only used as a checklist to confirm that the topics were discussed thoroughly. The interviewer was mainly listening to the interviewee and interrupted only to ensure, that the discussion remained within the scope of the research. The interviews lasted around 40-60 minutes and were arranged with one or two representatives from the organization and one or two researchers in a face-to-face meeting or over a video call. The interviews were either tape-recorded, or extensive written notes were taken during the interview to ensure a sufficient level of recorded data.

3.2 Data Analysis

The applied analysis method in this study is the cross-case synthesis, which is used in multiple-case studies [²⁹]. The key to this analysis is to provide concise findings by analyzing multiple cases separately and combining them into one case. In this study, each of the hackathons of each organization is analyzed as one case. The results are formed by combining all of the cases, which minimizes the number of outliers and gives more strength to the findings.

The first step of the case analysis is done by constructing matrices based on the subsidiary research question [²¹]. For each case a matrix is constructed, categorizing the motivations, benefits, and challenges in the analyzed hackathons. The data from the interviews were then mapped into these matrices by using the recordings, transcriptions, and notes available. Each case matrix consisted of statements and their contexts, which were carefully analyzed, understood, and codified into a more concise form. For example, lengthy descriptions about how a prototype was taken into production was codified simply as New product, while more complex descriptions about the networks and partners, or occurrences within the event, were given longer description, such as Development in ecosystem with participants from different environments (Case C) and Views about what can be done and own people starts to see opportunities (Case F). Codification is used as a tool in order to demonstrate the core messages in the data and the wording of the codified outcomes were taken from each case separately. This way each of the cases were analyzed separately, minimizing involuntary synthesis before all of the cases were analyzed.

After the cases were analyzed, the matrices from each case were then deconstructed into a final, synthesized matrix, where the findings were distilled through combining similarities and anomalies [²⁹]. Each finding was categorized by abstracting them against all of the other findings. The initial set of categories was collected from one case, recognizing distinct patterns from the codified terms. Similar terms were synthesized under a broader category, and the rest were left as is. These patterns, or terms, were then taken to the next case and compared. Again similar terms were synthesized, and the codified, final term was broadened, if necessary, to fully describe all the used terms. This was done until there were no more cases to analyze and all of the terms from case matrices had been synthesized into the final matrix. For example, the codified result New product was abstracted as a part of New products, projects, and research, while the longer codifications, mentioned in the previous paragraph, were re-codified as Development in ecosystem (Case C) and Internal / organization development (Case F). The original data from the cases related to each term were then used to make descriptions and examples of these abstractions for the findings. By these actions, the effects of the environment and personnel were minimized in the categorization, but the results stay true to the original data. The data analysis and the order of execution are further visualized for clarity (Figure 1).

Figure 1: Cross-case synthesis execution 

When the complete categories were finished and the findings about the hackathons synthesized, the categories and their descriptions of both, benefits and challenges, were compared to the categories of open data benefits and drawbacks. Utilizing especially the descriptions with similar or identical topics and keywords, the corresponding categories from both fields were taken under further scrutiny. The open data benefits and drawbacks from the literature review were used as the base categories, where the hackathons were mapped into. This allows this research to discuss and bring forward the different aspects of open data and compare the hackathons to them, finding similarities and differences between the practices. While this method of comparing categories is unstructured and superficial, it allows some insights into the fundamental differences of these methods as a tool for innovation in the different business domains. From these differences, conjectures can be drawn to explain, why one method is somewhat popular in the industry and the other one is not. However, it should be noted that these conjectures are not facts, but results from qualitative case data and a crude comparison between two practices. They can be used as the basis for future research in a form of provable hypotheses, but not as definite statements.

4.1 Benefits

From the analyzed hackathons, four benefit categories were present in the majority of the cases, with three existing in basically all cases (Table 3). The new ventures in a form of new projects, products, and research were reported in every case, which indicates that each organizer got something practical out of the event. For some organizers, this was more superficial or incremental within the organization, while to others this meant completely new products that were implemented into business quickly. The second popular benefit was changes or potential changes in the organization. The personnel organizing the event and also the visiting employees describe the hackathon as an eye-opening event, an example that agile processes can be used even in their organization. It also enabled the different departments to work together for a common goal. The third major benefit is the change in the current ecosystem and how the event molded or can mold the existing ecosystem through new partners and collaboration and the use of external professionals to enhance development with an outside point of view.

In Table 3, each case is presented as one column, using an x to demonstrate, that the category was mentioned in the case interviews and was seen as a benefit in the case. After the table, each category is explained with descriptions and examples derived from the case, to give context to the category.

Table 3: Benefits from hosting for case organizations (x = was mentioned in the interview) 

New ventures: projects, products, and research combines the new ventures that were achieved. While the main goal of many organizers was to achieve new products, the level of which this was realized varied. In a couple of cases, such as Case G, Case D, and Case J, the ideas and prototypes remained at a concept level and were not yet implemented into business. Some of the organizers started piloting with one (Case H) or more (three in Case F) proof of concepts and in Case B, one prototype had been proceeded into testing in customer site while others remained in discussions. Few cases had already moved even beyond testing: in Case C the prototypes were converted into research projects with the teams, in Case E a product was implemented and was generating revenue at the time of the interview, and in Case A the product had been ready to launch but was withheld because a competitor launched similar product before them. For some organizers, the yield was even more than just isolated products: In Case I one of the major outcomes was that they found new models how to connect with external developers and in Case D it was reported, that new collaboration and projects were being set up after the event, without a clear connection to the prototypes.

The category Internal / organizational development was mentioned in nine cases out of ten, perceived on different levels. On an individual level, an energizing and an eye-opening impact was reported in multiple cases (cases C, J, B, and G), the employees were excited and enthusiastic about the new ways a prototype can be developed. The hackathon also brought about development towards a more open culture within the organization (cases J and E) and allowed everyone the chance to innovate and see different opportunities outside their current positions (cases G and F). A change was also reported in the communication between the departments in cases C and D: through the event multiple departments were activated, the information flow strengthened, and the feeling of togetherness was increased. The increased communications within the organization also allowed the organizers to take a new approach to in-house product development, changing the process how innovation has been done within the organization (cases A, I, D, and C).

Ecosystem development covers a multitude of actors from external developers to other partners and collaborators within the industry. In multiple cases (A, B, D, I), it was mentioned how hackathon can be used to meeting new partners and suppliers through the common interests. Instead of a random sampling from the field, the event can be used to focus on a specific group of external developers (cases A, C, and J), who would be difficult to meet otherwise, and also the organizer can see, what is currently going on in the field (Case F). The event was also used to market the organization: in Case B the hackathon organizer wanted to work with start-ups and the event was used as a marketing tool to show how. Lastly, in some cases (C, F, J, B, and H) the hackathon was seen as a method to create a new form of partnerships, where a larger portion of development and conceptualization would be shared, gaining an edge through a large network of partners and individuals.

The category Feedback and communication is used to describe the effects of face-to-face meetings in the event and also the new knowledge that was brought inside the company by the external developers. In cases E and I, the organizers reported usable feedback about their data and how could the data be improved in the future. On the other hand, through discussion the organizer was able to offer a view of the industry, what are the current issues and challenges (cases B and C) and the participating teams were able to view the issues from an external point of view and apply new technologies and solutions to the problems (cases E and F). Finally, in Case G, the organizer reported that the hackathon was used as a marketing material, giving empirical talks based on the event and company strategy.

The category Motivated system development refers to three stages: before, during, and after the event. Before the event, the organizer had to make sure, that the technologies, systems, and data were sufficient for the external developers, and the event offered a suitable deadline (cases F and I). During the event, in cases A and C, the teams were helping the organizer to develop and enhance their systems, while there was no direct business benefit in these incremental developments. Additionally, the organizers got feedback about their systems from an external point of view, which allowed them to realize faults and points of improvement in their systems (cases F and I). After the evet was over, the organizer was left with new motivation to improve the systems even further (cases I and J).

Accelerated R&D refers to the possibility of increasing the velocity of the innovation process. In multiple cases (A, C, E, G) the organizer was impressed by the speed in which the prototypes were built and their overall quality. For example in Case C, a concept was thought of for three years, it was developed for half a year, and implemented by outsiders within two days. In Case F it was also highlighted, that the engineers had to crystallize their results within a couple of minutes and this was seen as a significant improvement.

Visibility was mentioned as a positive benefit since the events had been noticed by a multitude of possible partners, which had then contacted the organizer and referenced the hackathon as a point of interest (cases B, D, E, and G). In Case D, the organizer highlighted the fact, that the hackathon was used to show partners and other organizations, that they are open to working in different ways.

Some organizers (cases A, C, and H) reported that they had built an Archive of ideas and concepts from the event. They were not capable of implementing all of the concepts immediately, and the unused concepts were stored with the intention of implementation whenever possible, depending on markets and available resources. However, it was mentioned in case C, all of these ideas did not come from the hackathons, but from within the organization when preparing for the event or after it.

The least mentioned category was Recruitment, where the organizers used the event in order to recruit new talents. For instance, in Case D the organizer was recruiting software developers for their subsidiary and was also building their brand for the possible experts looking for new positions.

4.2 Challenges

The challenges of hackathons did not offer as uniform results as the benefits (Table 4). Eight out of ten organizations reported that they had issues with their own internal processes or in general in the organization of the activities when hosting the event. This is a major issue, while not an unexpected one since the large organizations tend to have more detailed and defined management processes. Introducing agile development into these processes was sometimes considered problematic, although in some cases it was one of the major benefits as well.

Some challenges correspond to the benefits, and imply a very polarized view of the hackathon experiences, but also indicate the motivation to improve based on the challenges. Challenges such as the lack of communication and technological issues, or resource and cost management are likely to be not caused by the hackathons but become issues because of the organizer’s own internal need for development. However, some challenges emerged because of the event and did not have a direct link to the organizations, such as preparatory actions and scheduling.

Table 4: Challenges in hosting an event (x = was mentioned in the interview) 

The category Process and organizational issues was mentioned the most. The category refers to the issues noticed in the organization, how the culture of the organization was not suitable for hackathon as an innovation method, or the processes within the organization did not support the agile development principles. In cases C, D, G, and J, the culture was mentioned as a major hindrance, it was difficult to engage individuals in the development and management to take the projects forward. Another issue was the innovation process within the organization, mentioned in cases A, B, D, and F. After the hackathon, the projects are supposed to be taken under further development and commercialization, but in these cases, getting the project through the project office and securing funding was a major hindrance and a cause behind losing the momentum. Additionally, in Case H it was mentioned, that the number of business secrets, related to the functions of the organization, were seen as a limiting factor, losing the competitive edge was seen as a major challenge.

The category Resource and cost management represents the issues with funding and the strain on the employees, who are running the day-to-day business in the organization, even though in Case B the interviewee mentioned that in hindsight, focusing resources to the event was somewhat irrelevant, since the event only served as a starting point. In cases B, C, D, and F, the organizer reported a shortage of manpower to make the preparation for the event, especially because of the scale of the event: larger event requires more people to prepare and manage it, in addition to the day-to-day responsibilities. On the other hand, in cases C, F, G, and J the funding was seen as an issue and an extra cost of the event. In Case F, one of the prototypes was discarded because of the scale it needed, there was not enough budget or other resources to create a project. In addition, in Case G the funding was even more difficult, since the nature of their business leans towards design and physical products, the benefits from hackathons could not be measured in direct sales, which made it difficult to justify funds for the winner and other costs.

The category Difficulty of communication refers to the issues with the language barriers (e.g. lack of English skills from presenters in cases E and H), whereas some teams were just challenging to contact in general. In cases I and J, the lack of communication and mentoring during the event was noticed, which caused a mismatch between the initial goals and the solutions. This could be an issue in public hackathons because the teams are not as controlled as they are in industrial events, but similar problems were mentioned in Case E, that there were not enough updates on the progress of the prototype, which also led into a mismatch of interests. In Case D, the interviewee highlighted the fact, that the communications have to be planned beforehand, a sentiment also mentioned in Case J.

The category Technological issues describes the practical issues during the events. In some organizations the access to data was limited and also there was a lack of suitable data, which caused the solutions to drift into a certain direction, while the organizer would have wanted more variance (cases A, I, and E). Another key issue was noticed with the current systems, the organizers did not have sufficient IT systems to support the hackathon, which limited the information flow to the teams and the prototypes were limited in this sense (cases E and H). In Case F it was mentioned, that because of interfacing problems with some teams, it was necessary to postpone the development altogether.

Challenges in preparation were noticed by the organizations: they recognized issues in preparation and were able to cancel out the possible risks and pitfalls. When preparing for the event, the challenges were formulated and resources were prepared for future development. Many organizers reported issues that were natural for a first-time organizer: adjustment of time, technology, resources, and winning criteria, as well as the selection of teams and participants (cases A, C, and F). Another issue was recognized in the environment and the scope of business of the organizer: the hackathon is not an optimal tool for every organization because the themes can be intangible, and the environments constrained, which can limit the level of innovation that the external professionals can bring into the event (cases C and H). Finally, in Case D the publicity was mentioned as a high risk. If something went wrong with the preparations, the combination of the visibility of the event and social media could cause irreparable damage to the company brand.

Some organizers experience issues with the Team commitment, which refers to the readiness for follow-ups after the hackathon. The teams were reported to be slightly disappointing to the organizer, because of few reasons: either the teams were not ready to commit to proper collaboration and only wanted to sell their concept (cases B and C), the developers were not that interested in the data of the organizer (Case I), the size and scale of the team and the backing organization was not sufficient for the scope of their prototype (Case H), and the teams did not change their visions based on expert comments or were unable to present their concept sufficiently (Case H).

The category Schedule did not appear as much as was expected: Since the hackathon events tend to last for a couple of days, it does not offer enough time to develop a highly refined solution for the complex problems and objectives. In Case H, the scheduling was especially difficult since the environment was changing during the event, and finding a date that could work for everyone involved was challenging. In cases C and D, the organizers were even contemplating that in hindsight there could have been multiple things that could have failed because of the tight time frame.

The Ownership of the product inside the organization is an issue since someone has to be responsible for the further development and the rights for it. Because of the common deals beforehand, where the property rights are agreed upon, the rights were an issue only in Case C, where two companies did not agree with the organizer. The results in an industry, which relies heavily on the visual design (Case G), allowing anything to leave the organization was not an option which was difficult situation for the teams. In Case E, it was highlighted that the ownership within the organization is extremely essential, and even the profitable concepts would be buried if the ownership is unclear in any way.

The events that were held as a curiosity show that the Lack of follow-up is a challenge. Every event, where something was created but nothing was taken forward, fundamentally only wasted resources. For instance, in Case G, the results were shared internally, they did receive positive comments, but nothing else happened. This happened only in two cases and in situations, where there were severe environmental or organizational issues preventing the development. In Case J, the interviewee mentioned that the advertisements of the prototypes developed in their hackathon were seen in other events and seminars; the teams were developing the prototypes themselves.

5.1 Benefits from Hackathons and Opening Data

The comparison of benefits is done by comparing the results of the systematic literature review about open data benefits [¹³], described in section 2, to the results of this research. In this comparison, the six business benefits found from the literature are used as the base categories, and the hackathon benefits are then placed into the categories based on their similarity. This means, that one benefit of opening data may have more than one corresponding benefit from hackathons. The same method is then used when comparing the recognized issues in opening data and hackathons.

Collaborative actions: There were similar elements in the collaborative actions emerged from opening data and hackathons. With open data, the collaboration between the consumers and the companies was triggered and enhanced through the data, allowing communication, feedback, and information sharing [⁷]. Hackathons, on the other hand, offer significantly more relevant communication and feedback, since the participants are familiarized to the environment, they are experts from relevant fields, and the solutions and development can be discussed face-to-face during the event (Feedback and communication). Because of this, the insights towards products and services are more useful and significant [¹⁸]. Opening the data can bring incremental development through two-way dialogue, while hackathons can introduce new technologies and techniques to the organization in a controlled environment and promote collaboration with teams, current partners, and also with new, or potential, partners (Ecosystem development).

Competitiveness: Easily measured effects towards competitiveness are not found from opening data since the nature of the initiative demands that the data is given away. Unlike hackathons, open data does not necessarily offer any new ventures to the organization, but enables outsiders to use the data for their own purposes, while the hackathon organizer was able to gain new ventures in every case in this research (New ventures: projects, products, and research). In some cases, the transparency and information sharing is a tool that the smaller companies use to enhance their competitiveness against the larger organizations [²⁶]. However, sharing data without limits requires a suitable balance, since even one mistake can have negative impact or even prove fatal to the data provider [²]. To counteract this, the control in hackathons is necessary for the organizations to withhold their data and direct the innovations with the freedom to choose their participants and data users, which does not expose them to the competitors. There is also the aspect of recruitment, (e.g. [²²]) since opening data does not bring new people to the organization, except for some rare, special cases, while hackathons allow the creation of personal contacts (Recruitment).

Ecosystem-wide engagement and communication: While opening data may increase the communication inside an ecosystem, the hackathons take this further (Ecosystem development). When data is opened, the data provider has limited control over the data usage and discoverability. Through hackathon, the participants are screened and motivated to use the given data in the event to nurture their concepts and ideas further. The organizer can monitor the participating teams, make suggestions, and even develop projects with multiple teams. By meeting with the developers, the communication is more coherent and through control, both sides can create something practical, while with open data the communication is limited to data [¹¹]. Through hackathons, the roles in the ecosystems do not change, but opening data may require additional changes to the data provider, and also for other organizations in their ecosystem [²⁶]. It was also noticed that the hackathons offered new partners and suppliers for the organizer, widening their ecosystem, which was not mentioned as a benefit of open data.

Innovation and development: The innovations based on open data, incremental or radical, can be made by a variety of developers because of the availability of open data. However, in the scope of one organization, the innovations based on open data may or may not aid the data publisher, since the data has already been opened and the innovation is done by someone else. The control of the data, in this case, is crucial in order to oblige the innovator to collaborate with the publisher. Without control, the innovation, radical or incremental, is difficult to commercialize by the original manufacturer and the data provider. The hackathons, on the other hand, offer more practical cases, because the participants are familiarized with the industry and the systems, so the innovations and development can be focused further and it serves as a base (Archive of ideas and concepts) to relevant innovations that the organization can use [²⁴]. With hackathons, the rights and ownership of the created solutions are agreed upon beforehand, but with innovations based on the open data can be difficult to manage, as was mentioned earlier. Hackathons were also linked to the developments in the innovation process (Accelerated R&D) within the organization, an aspect that opening data does not promote.

Internal change: There is quite the contrast between opening data and organizing a hackathon in a sense of internal development, since opening data allows the organization to see flaws and issues in their processes, while through hackathons the company can develop their systems (Motivated system development) and processes in a controlled manner towards more agile development (Internal / organizational development). Another contrast is in the scope of cultural change inside the organization. When data is opened, the change is reported as being a necessity [¹⁷], while with the hackathon it was found as an additional benefit born from enthusiasm, although not in every case. In a hackathon, the departments are creating new collaboration and increasing communication between themselves voluntarily, which can increase efficiency and change the ways things are done within the organization.

Public image: The visibility from hackathons is limited to knowing about the event and the event description, but the outcomes are not always disclosed (Visibility). Opening data, on the other hand, can be utilized by anyone and through use, the data gains more visibility. Visualizing opened data for consumers is a powerful method to deliver useful information to environments and adds value to the transaction. Additionally, depending on the industry, companies with a large consumer market can use the publicity and transparency to stand out from competitors [²⁶].

5.2 Challenges in Hackathons and Opening Data

The comparison of drawbacks and challenges is not as straightforward as the comparison between the benefits. Some of the challenges in hackathons are about the issues from third parties, such as team commitment, or issues born from the very nature of a hackathon, such as a schedule for the event, preparations, and follow-up, issues also recognized in [²⁴]. Because of this, they do not correspond well with opening data.

Decrease in efficiency: One major issue with opening data is the duplication of processes and ownership of the data. As it was mentioned, the efficiency of opening data can hinder, because the processes are not updated and the organization is using duplicate processes for the same tasks [¹⁴]. With hackathons the nature of this issue is different since the organization does not commit assets; much of the development is done by external professionals and ownerships are negotiated beforehand. After the event, the follow-up is done by following the existing processes. If the follow-ups require creating some changes in the organization, the changes are incremental and manageable and they do not require large changes in the current state, even though it was seen as a challenge (Process and organizational issues). Another issue is the ownership of data: if the ownership is not clear, for example, the data is owned by a third party, this can make opening data ineffective or even prevent opening of data completely [¹²]. In hackathons, the ownership of the prototypes and further, the product ownership was seen as a major issue (Ownership), since clear ownership would prevent the prototype from further development, and would not be done.

Increased costs: The costs of opening data originate from collecting the data, data visualization, and training, but also from managing the systems. Additional costs may also be caused by the documentations, metadata, and other additional materials that could be necessary for the data usage [²]. In hackathons, a major issue was the availability of resources and funds for the event (Resource and cost management), in order to host the event, but also to move the prototypes forward into products. However, in hackathons, the data does not need to move outside the organization and the participants are on the premises, which allows the organizer to use the pre-existing systems, so the technologies do not have to be developed to very advanced degree. While the event does cause a strain on the resources, in most cases it is only a short and temporary need, while opening data would require more financial, technological, and ideological long-term commitment.

Public data increases problems: While there is not a category from the hackathons that matches directly with this category, there was a mention in Case D in the category Challenges in preparation. If the hackathon is not sufficiently prepared for and something goes wrong, the combination of publicity and social media may cause issues to the organizer and their brand. This issue seems as a minor hindrance since open data can be used to purposefully misunderstand facts, used against the company, or used to invade privacy, which can affect the brand even more [³¹]. Hackathons also had an issue with the communication during the event with the teams (Difficulty of communications), causing the organizer to receive solutions, which did not match the initial goals. However, during the hackathon, the lack of communication can be an issue, but crucial misunderstanding and misuses can be prevented through control, which is not an option with the open data.

Required change: In the literature review and in this research, the processes and culture of the organization were deemed an issue (also [¹⁴], [¹⁷]), but also one of the most potential subjects of development. With opening data, the amount of change for the organization and their ecosystem was noted to be substantial, requiring changes in the policies, focus, business models, and ideologies and not only for the organization opening data, but possibly for their partners as well. While opening data requires a substantial commitment to the open data initiative and changes in the organization, the hackathon is contained within the organization and rarely requires changes to the organization or their ecosystem, while they could be heavy on resources (Resource and cost management). This decreases the opposition and makes the changes more relevant and fits better into the culture of the organization. Applying a process for opening data in an unsuitable culture can lead to harmful actions within the organization [²], especially if opening data is applied forcefully, while in hackathons the culture change emerges from the individuals, not from the strategy. In addition, the hackathon may require some technological upgrades, as was noticed in half of the cases (Technological issues), but opening data requires a whole new set of systems.

5.3 Conjectures from the Study

In conclusion, the benefits and drawbacks of applying hackathons as an approach to open and share company data are summarized. When comparing opening data to hackathons from the viewpoint of the organization, there seem to be multiple benefits, but scarce evidence about why opening data could be more profitable. Hackathons offer:

More control over the whole innovation process than opening data. The organization that organizes the hackathon has control over who participates in the event. This allows control over the teams and ultimately the solutions they are going to develop during and after the event. The innovation can also be influenced by the event because teams can communicate with the organizer and their experts in order to gain insights into the industry. This decreases the risk of misunderstandings and the misuses of data.

Practical business cases and enthusiasm towards improving the culture within the organization. Open data as an initiative has its largest drawbacks on when the organization is publishing data in order to generate new business. The hackathons in this research have shown, that through the events the host gets something new in the form of projects, products, and/or research, while through opening data this is not guaranteed. Some of the organizers are already reporting profit from these innovations, while some organizers utilize the results as incremental development. It was also noticed, how opening data requires systematic change, while hackathons inspire and motivate the employees to change the culture through actions, not because of a strategy.

Engagement and communication with educated experts better than through open data. Data is a lacking medium to hold a discussion or other form of communications, but it can be used as the base for discussion. In order to engage the experts and developers, it seems to be more beneficial to meet face-to-face and discuss the topic in a suitable environment. This way the communications are clear and misunderstanding can be resolved quickly, even though this was not a complete success with hackathons either. In the event, it is also possible to motivate the experts and assess their skills, which may even lead to new recruitments.

Fewer changes in day-to-day processes without external changes. Unlike opening data, organizing a hackathon does not require a sustained strain on the internal resources, since the event takes only a few days and the collaboration with the teams does not change anything within the organization, since the teams are at most treated as another partner. Working with external teams also allows flexibility to the organizer, since they do not need to commit resources and systems as much as they would need to with opening data. The collaboration with the teams may require some changes to the software systems, but they are usually incremental changes and do not require new systems. What hackathons also offer when compared to opening data is that the necessary changes are confined to the organization and it does not affect the existing partnerships in a way that opening data might change the environment. Opening data also requires commitment and if the company culture does not adopt or adapt to opening data, the changes can be difficult and possibly costly to either roll back or enforce.

As mentioned, when comparing the hackathons and opening data, in this research opening data is seen to have only one tangible reason, why it seems to be the more efficient solution. That is:

Open data attracts more public visibility than hackathons. The hackathon is, by nature, a closed event, usually accessible only through a screening process. Open data, on the other hand, is by definition accessible to anyone and it can be used for any legal objective imaginable. In hackathons, the organizer can limit the scope of the solutions by determining goals and limitations because the event is a competition between teams. These aspects show, how open data has the potential to reach any number of individuals and serve as a free advertisement to the data publisher; the data can be also used more freely in order to develop new products and services by consumers. While hackathons offer strict visibility to partners, open data allows widespread publicity through data. Although, depending on the nature of an organization, the type of visibility that a hackathon offers may be more suitable for their goals. It was also mentioned, that the visibility of open data can be a benefit, but it can also be a drawback, depending on how the data is used and by whom. Because of these reasons, even the extended visibility through open data is something that organizations may not want and they could prefer a more limited scope of advertisement.

While this research cannot be generalized to public institutions and the scope of this research is private organizations, some lessons learned can be drawn from here. Opening data is another strategy to allow access outside the organization, but the critical control and harvesting of benefits are more efficiently achieved through the hackathons than the blind opening of data, possibly because of the planning and organizing that hackathons require. It would seem that in comparison to open data, hackathons are more beneficial to companies because the return of investment can be calculated more accurately and there seems to be trust that hackathons will deliver benefits.

5.4 Limitations of this Study

In qualitative research, a multitude of risks towards the credibility of the research exist [²³]. For instance, the researcher bias is troublesome in this type of research, where the researcher can affect the interviewee through active or passive actions. Such instances can occur when constructing the questionnaire while collecting the data and during the data analysis activities. In this study, this issue was taken into account by using two researchers to construct the questionnaire and consult the organizations, which were developing and arranging hackathon events, and by using four researchers representing four different research groups in the analysis and publication of the data. In addition, the interviews were held in the native language of the interviewee in order to allow informal discussion, to ensure that the interviewee was able to speak their mind, and to ensure that the interviewees understood the questions correctly. Finally, the data were analyzed by using cross-case synthesis method, which allows more robustness because of the abstraction of multiple cases [²⁹]. In general, the principles defined by, for example [⁸], [²⁵], were followed to enhance the trustworthiness, rigor and the overall quality of this study. In any case, these types of qualitative studies cannot offer any generalization in a mathematical sense, but rather offer a generalization of observations, which can be applied as guidelines or considerations for best practices when outside of the study scope [²⁷].