The Art of Crowsourcing
April 01, 2016
The United States Library of Congress uploaded over one million historical images to the digital Commons via Flickr with no copyright assigned to any of the images. The idea behind the gift of high resolution, high quality historical photos was that in exchange for the unsorted photos, Flickr users would eventually organize and tag all of the images, providing metadata for the content that would otherwise take years of efforts and large amounts of government funding if the Library of Congress hired a staffer for the task.
Similarly, Google's reCaptcha system asks users to enter a piece of garbled text or blurred set of numbers into a text box in order to prove their humanness to the computer. This test of being human is a coordinated effort by Google to get consensus on what the text says after a computer has failed to decipher it by way of optical character recognition tests. reCaptcha accumulates over 150,000 hours of labour a day from the brief seconds of effort exhausted by Web users as they try to login to their respective networks while proving they are in fact human.
The collating efforts given to the Library of Congress and the login attempts via reCaptcha are both examples of crowdsourcing, a term coined by Jeff Howe in the article "The Rise of Crowdsourcing" from Wired magazine in June 2006. Crowdsourcing is, essentially, when "companies operate by broadcasting problems or challenges to the crowd. Individuals in the crowd offer solutions to these problems and post the solution back to the online commons" (Brabham 2008 1124). Crowdsourcing takes advantage of the technology of the Internet, providing "unprecedented levels of collaboration and meaningful exchanges between people from every imaginable background in every imaginable geographic location" (Howe 14). Crowdsourcing takes advantage of people's creative energies, which is as "endlessly inventive as it is infinitely renewable (Howe 177). It affords the opportunity to be creative to everyone, professional or amateur, regardless of geographic location, age, race, or gender; as the infamous New Yorker cartoon suggests, "on the Internet, nobody knows you're a dog". Crowdsourcing is "design by democracy" (Howe 2).
PETER STEINER - The New Yorker magazine (1993)
Crowdsourcing succeeds because of the idea that every human being has his/her own unique experiences and histories, thus we all have our own unique sets of knowledge from which to solve problems (Howe 139). This massive pool of knowledge and skill provided by the people working on a crowdsourcing task, in theory, should always be able to outperform a group of experts. In dumbed down terms, it's like the old adage of a million monkeys typing on a million typewriters, eventually they will write all the works of Shakespeare, despite not being, in the case of crowdsourcing, paid professionals.
The idea that crowdsourcing is a democratized human-powered supercomputer seems quite utopian, and in fact, has been proven to be somewhat idealistic. The framework for this utopia, however, is certainly in place and ready to accommodate each and every person willing to contribute to the cause, but because of the digital divide, this isn't quite the case, yet. The digital divide is the invisible dividing line defined by those who do have consistent high speed Internet and those who do not. This divide falls primarily between developed and developing nations, but even within the developed nations there are factors such as remote and rural living conditions that cannot support high speed Internet structures, and low income families and individuals that cannot afford home Internet access. Given this digital divide, the crowd is currently primarily comprised of white, middle to upper class, English-speaking, higher educated men in their late twenties (Brabham).
That said, the crowd is the primary organizing force behind crowdsourcing (Howe 100). Howe points out that successful crowdsourcing comes out of "a deep commitment to the community" (15) and is an organic process that cannot be created by businesses; it is the people in the crowd that determine success (13).
The people in the crowd are often monetarily uncompensated for their efforts—no PayPal deposits are made on Google's behalf when a user enters their answer into a reCaptcha. Crowdsourcing projects like Wikipedia work off of a non-profit mentality, expecting volunteer labour from the crowd to help build the repository of encyclopedia entries. Commercial projects, such as iStockphoto—a crowdsourced stock photo agency—offer financial compensation to crowdsourcers when their images are sold through the iStock.com Web site, but even that compensation is a pittance of what a professional photographer would make dealing directly with the client. Despite the small individual profits, there are iStockphoto contributors earning in excess of 10,000 dollars a month for their efforts (but this is not the norm) (Howe 27).
Crowdsourcing's formula for success comes from the low cost of production (computers and high-speed Internet access are all within reach of the average middle class income family), underemployed talent (in large part due to the collapse of Wall Street in 2008), and the online communities of like-minded people (Howe 5). The members of the crowd give away their free time and are not motivated by money (Howe 29) and this surplus of free labour has professionals in industries which crowdsourcing has infiltrated upset. Businesses, like stock photography companies, are built on scarcity, and this scarcity is being undercut by crowdsourcing (Tryon 142). Professional organizations like the Society of Graphic Designers of Canada (GDC), and the American Institute of Graphic Arts (AIGA) have spoken out against crowdsourcing (often lumped in with speculative work). Professional artists argue that speculative work is unethical because artists and designers often end up working for free "with an often falsely advertised, overinflated promise for future employment; or are given other insufficient forms of compensation" (nospec.com). An ideal analogy for speculative design work is this: If you were building a new three-storey house with multiple bathrooms, would you hire a plumber for each bathroom and promise the best plumber the contract for the kitchen, while additionally not paying the losing plumbers? This notion seems ludicrous to most people because you need to pay a professional for their time, energy, and materials. While there can certainly be an exploitative nature to design contests such as these (best showcased by 99designs.com et al), they are not the ideal scenario for crowdsourcing. The most effective crowdsourcing attempts are those that succeed from a multitude of minor tasks from the crowd, where the effort is minimal, taking minutes, if not seconds, instead of hours or days, as seen in the case of design contests. Additionally, not all crowdsourcing falls under the corporate umbrella: Wikipedia is the biggest and best known example of a not-for-profit crowdsourcing endeavor.
Successful crowdsourcing ventures, as previously stated, must break large tasks down into small parts to ensure that the task is not too daunting or overwhelming for the crowd. Additionally, the challenges put to the crowd must be complex; simple tasks do not need a crowd to solve them (Howe 143) (after all, how many people does it really take to screw in a light bulb?). Because the tasks given to the crowd are complex and challenging, they are often appealing to the crowd, and the crowd attracts both amateurs and professionals alike (Howe 28). While the crowd may be diverse in a multitude of ways, the crowd must also be qualified; a random group of riders on a bus won't likely be able to trump the problem-solving skills of a group of aerospace engineers, unless that bus happens to be shuttling employees from the NASA parking lot (Howe 143). This group problem-solving aspect of crowdsourcing capitalizes on the innate social nature instilled in the human race (Howe 14) and ultimately adds to our cultural and intellectual capital (16).
Art & Design for profit
Threadless is an American company from Chicago that runs weekly design contests on its Web site, threadless.com. Each week the site receives approximately 1000 designs from its 600,000 members. Members vote on the designs they like best and wish to purchase as a T-shirt. Six of the top one hundred designs are selected and printed each week. Winning designers are paid out up to 5,000 dollars and given a 500 dollar credit for the site (threadless.com). This large payout was not always the case: designers originally received a complimentary T-shirt of their design for being successful. Threadless sells over 90,000 shirts a month using this crowdsourcing model. Threadless does very little work to profit; it hosts the site for the community to upload and vote on designs, and it produces what the crowd tells it to. "Threadless isn't really a T-shirt business. It sells community" (Howe 6).
The crowdsourcing community provides the talent, and it ranks and filters the results, determining what should (or should not) be sold (Howe 223). Ironically, Threadless' most popular design ever is the "Communist party" T-shirt.
The best-selling T-shirt design on Threadless.com
Another successful crowdsourcing Web site is istockphoto.com. Originally, iStock was a photo-sharing site for designers to upload their leftover (or stock) photographs. Bruce Livingstone originally conceived of the site in 2000. As popularity for the site began to increase, Livingstone added a charge of 25 cents for each downloaded royalty-free image, undercutting the professional agencies by 99% (Howe 7).
Stock images are the "little white lie" (Howe 7) of the publishing industry. They are nothing more than a collection of unused photographs from other projects that are licensed for re-use. While iStock is certainly taking away business from the professional, it is also democratizing a product by allowing the amateur to participate in the industry. And at the end of the day, the profits made from stock photography are surplus to the core job of the photographer, and crowdsourcing, in this case, can't eliminate the professional photography market completely.
Daren Brabham conducted a survey of the iStockphoto community—iStockers—to see why they participated in crowdsourced stock photography. He found, that despite the fact that most iStockers made little to no money from supplying the site with images, they continued to participate because of the community (Brabham). iStockers "generally felt that the iStockphoto community is one big family..." (Brabham). Further, iStockers felt proud of being part of the community and that participation gave them "a sense of accomplishment, competence, and effectiveness... [and that participation was] an important activity for themselves" (Brabham).
Art for Art's Sake
Howe originally saw crowdsourcing as a model for business back in 2006, but artists have taken advantage of the benefits afforded from crowdsourcing without working within corporate structures. Crowdsourced art is "the practice of using the Internet as a participatory platform to directly engage the public in the creation of visual, musical, literary or dramatic artwork with the goal of showcasing the relationship between the collective imagination and the individual artistic sensibilities of its participants" (Literat 2964).
Crowdsourcing isn't anything new in art. In the 1920s, surrealist artists applied the "exquisite corpse" technique in which artists would contribute to a larger piece of work without seeing the previous contributions (Literat 2971). Even centuries ago, Japanese poets were practicing the art of renga, a form of collaborative poetry. The first renga poet would write three lines of a five line poem and pass it off to another poet who would finish the poem. From there, the last two lines would be used by a third poet as the basis for the first three lines of a second poem, and so on, creating a poetic daisy chain (Chimero). The structure of the poem worked similarly to today's crowdsourcing: the tasks given were small but challenging, there was an inherent structure to the task, and it took relatively little time.
Current artistic crowdsourcing projects stem from the Happenings of the 1970s, which were focused on "distributed creativity, gift economies" (Browne 34) and required a physical presence in place and time; the audience was a necessary component and participant in the works (Literat 2963). Today's crowdsourced art extends this idea to the global community through the Internet.
Today, every artistic medium seems to have found a way to crowdsource. In late November 2010, Tim Burton authored a short story titled Cadavre Equis with the help of the crowd. Participants were encouraged to tweet sentences to help continue the story along by adding the hashtag #burtonstory to the end of their tweets. Burton would select the tweets that would be added to the narrative.
Casey Pugh created the feature-length and Emmy-winning film Star Wars Uncut with the aide of nearly 1,000 contributors from the crowd . Pugh divided the original screenplay of George Lucas' Star Wars: A New Hope into small scenes and allowed the crowd to choose the ones they wished to reproduce. The resulting film is a mashup of puppetry, cartoons, real actors, and more. Pugh is currently in the midst of producing the sequel, The Empire Strikes Back.
A more traditional artistic approach to crowdsourcing is the project by Peter Edmunds entitled Swarmsketch. Swarmsketch has been running for 3857 days as of this writing, and Edmunds states that "Swarmsketch is an ongoing online canvas that explores the possibilities of distributed design by the masses". Each week a randomly selected search term becomes the topic for the crowd to sketch. The crowd has a week, or 1,000 'brush strokes', whichever comes first, to complete the drawing. Like more traditional/corporate crowdsourcing ventures, each user contributes a modicum of time and effort adding a small line (just a few pixels) to the piece. The crowd self-regulates the work by voting on the quality of the line within the work; the darkness or opacity of the line is determined by the vote tally (DeVun). "The results are something akin to the unholy union of a Cy Twombly and a Willem de Kooning drawing, and a very compelling argument for the mob's creative talents" (Grover).
“Packers vs. Cowboys”, swarmsketch.com
The utopia laid out in the creation of art by way of crowdsourcing isn't completely realized, however, Author Clive Thompson noted that
When the mob tried to draw a few simple pictures, it couldn't. Davis told it to draw a television, but the image never congealed. The group agreed that the tube should be represented by empty space, but it couldn't generate any other details. An attempt at drawing a face produced an even more shapeless mess. The only partially successful picture was a goat: At around 4,000 votes, it looked pretty goatlike, and at 5,000 votes the mob revised it to make the horns curvier. But after 7,000 votes the picture decayed into a random jumble of pixels, as if the group could no longer agree on what a goat should look like. Mobs, it seems, can't draw.
Thompson's reference to the mob is a response to Howard Rheingold's book Smart Mobs, a term used by Rheingold to, in essence, explain what Howe later termed crowdsourcing.
The Sheep Market is a piece created by Aaron Koblin, creative director for Google's Data Arts team. Koblin provided the crowd with a simple task: "draw a sheep facing to the left". He had the task completed by participants of Amazon's Mechanical Turk system. Amazon explains that the "Mechanical Turk is based on the idea that there are still many things that human beings can do much more effectively than computers, such as identifying objects in a photo or video, performing data de-duplication, transcribing audio recordings, or researching data details" (mturk.com). Mechanical Turk participants were able to draw up to three sheep and were compensated two cents for each drawing. The average drawing time took 105 seconds, which created an average wage of 69 cents per hour (DeVun). The Sheep Market collected roughly 10,000 images of sheep in 40 days. Koblin considered the artwork as "a comment on the unwitting participation of people in corporate crowdsourcing and the lack of creative jobs available to the crowd" (DeVun). It’s true that the Mechanical Turk is focused on business needs, not artistic endeavours, but one could argue that some business-related tasks, such as writing program code, could be seen as a form of art to an experienced developer who takes pride in the code he/she creates.
One of the 10,000 sheep from thesheepmarket.com
Participants of crowdsourced art (with the exception of those unwitting participants in Koblin's duplicitous corralling) join the crowd because of three main motivators: activation, authorship, and community. Activation provides the contributor with individual and collective agency in the artwork. Authorship creates democracy within the work as everyone has an equal opportunity to participate. Finally, community exists within the artistic process through the connectedness the participants feel working toward the same goal (Literat 2971). Collaborators accept these motivators as fair compensation for their efforts, but this crowd-created artwork does not fit the typical mode of creation: the sole artist toiling away on a painting or sculpture.
Crowdsourced art falls more in line with commercialized forms of art like Hollywood films or sold-out philharmonic concerts. The typical Hollywood film is created by hundreds of contributors, but ultimately it is the director who gets the credit for the film. Similarly, the philharmonic orchestra members are the ones creating the music heard in the concert hall, but the conductor is the one taking the final bow at the end of the evening (Literat 2974). This model for artistic production is much more akin to the corporate employer-employee relationship.
Ultimately the artist who creates the piece is the one who receives the credit because "it's the way the [art] assignment is conceived at the beginning that leads to good results" (DeVun). This statement holds true for all models of crowdsourcing, even open source software (which many perceive as the blueprint for crowdsourcing (Howe 8)) follows this tenet.
The Original Crowdsourcing
Open source software is, in some senses, the original form of crowdsourcing. Open source software runs under the GNU General Public License, which states that the software’s code can be edited in any way by the user and redistributed as long as it is kept under the same license. Additionally, no profits may be made from the sale of any part of the source code. The most famous open source software is the Linux operating system, which saw thousands of like-minded contributors work together to create a product that was better than what the professionals at Microsoft could create for their Windows operating system (Howe 8). Linus Torvalds, the initial contributor to Linux, stated that "most of the good programmers do programming not because they expect to get paid or get adulation by the public, but because it is fun to program" (Brabham). This notion of non-monetary compensation is a central tenet behind today's crowdsourcing efforts, and as Howe points out "crowdsourcing efforts have essentially just adopted an open source approach to making products other than software" (Howe 66).
Wikipedia is a great example of both open source and crowdsourcing; the software that runs the wiki site is open-sourced, and the content is created entirely by crowdsourcing. The site runs off of donations, but is not outside of the digital economy. Contributors to the online encyclopedia serve the digital commons for the greater good, but search engine companies like Google subsume Wikipedia’s content for capitalist means (Ligeon 20).
Crowdsourcing in the Factory
Marxist theory suggests that "'living labor' is needed to reanimate the 'dead labor' embodied in factory machinery. As a result of the surplus generated (Marx described this as a vampiric act), capital is able to roam, zombie-like, wherever it pleases" (Ross 25). This statement is an ideal reflection of crowdsourcing because the cognitive capital created by the crowd is needed to reanimate the computer, creating a surplus of information and ideas. Further, "the living labor is also allowed to range freely, choosing when and where to clock in, and whether to play along the way" (Ross 25). This affordance is created by the Internet; crowdsourcing would not succeed if it were not for the casual relationship the Internet provides to the crowd.
Howe states that community is the basis on which social capital is formed, and that "social capital provides the oil that keeps our economic machinery running smoothly" (Howe 117). He also suggests that "amateurs provide the crowdsourcing engine with fuel" (Howe 71), yet never mentions Marx through his 311 page book on crowdsourcing. It seems apparent that even Howe recognizes the underpinnings crowdsourcing has to the social factory.
While crowdsourcing fits the model of the social factory, it still manages to borrow heavily from the assembly lines of Fordism. Take for example the massive artwork of the Sheep Factory. One artist would have worked tirelessly for days or months drawing 10,000 unique sheep, but a simple, well defined task was given to the workers to complete individually without a means of communicating, much like the tasks given to the workers inside the Ford factory. To keep up production and high standards "you have to keep the assignment somewhat narrow in order to end up with something successful" (DeVun). Howe acknowledges that "breaking labor into little units, or modules, is one of the hallmarks of crowdsourcing" (49). Crowdsourcing is successful in replicating the Fordist ideology because of the Internet. "It is the rise of the network that allows us to exploit a fact of human labor that long predates the Internet: the ability to divvy up an overwhelming task—such as the writing of an exhaustive encyclopedia—into small enough chunks that completing it becomes not only feasible, but fun" (Howe 11). This last point is where crowdsourcing diverges from the traditional Fordist factory. As Marxist Franco Berardi states "classical industrial labor and specifically the organized form of the Fordist factory had no relation with pleasure" (Berardi, 84).
The Labour of Crowdsourcing
"In its broadest sense, labour is simply exertion of the body or mind" (Hesmondhalgh 276) and labour is not "equivalent to waged labor [sic]" (Terranova 46). Tiziana Terranova states that "the Internet is about the extraction of value out of continuous, updateable work, and it is extremely labor intensive" (48). Crowdsourcing can help in these labour efforts, but the general fear is that it leaves people vulnerable as resources for the maximization of corporate profit (Scholz). On the Internet we are all qualified labourers and corporations get "all the work without the worker" (Scholz). This volunteer labour is often seen as being exploitative of the crowdsourcing participant because they are often unpaid or underpaid, and at times unaware of the fact that they have benefited the corporation's coffers (Scholz).
These efforts can be interpreted as immaterial labour. According to Hardt and Negri, immaterial labour is "labour that creates immaterial products, such as knowledge, information, communication, a relationship, or an emotional response". All of these things are created through the efforts of participants of crowdsourced art: most obviously the emotional response through the viewing of the artwork. The production of knowledge, information, communication, and relationship can, and usually are, produced by and for the workers themselves.
Marxist, Maurizio Lazzarato, states that immaterial labour produces cultural content, among other things, and that immaterial labour "involves a series of activities that are not normally recognized as 'work'" (Lazzarato 133). The art produced through crowdsourcing is most certainly a cultural commodity, and given the minimal labour asked of each contributor and the pleasure they get from their efforts, surely they do not deem their tasks as work. Howe makes mention of "creative labor" (115) when he writes about crowdsourcing, which hints at the same ideas behind immaterial labour.
The significant factor of immaterial labour is that its primary product is a "social relationship" (Lazzarato 138), a central pillar to the success of crowdsourcing. This social relationship is referred to by Hardt and Negri as affective labour. Affective labour "produces social networks, forms of community, biopower" (Hardt and Negri 293) and "cooperation is completely inherent in the labor itself" (294). Cooperation, community, and social networks, again, are central to the success of crowdsourcing.
Returning to the idea of exploited labour, Howe states in his introduction to crowdsourcing that "given the right set of conditions, the crowd will almost always outperform any number of employees—a fact that companies are becoming aware of and are increasingly attempting to exploit" (11). This can be seen by Web sites such as 99designs.com that maintains a community of over 268,000 designers "from Berlin to Bombay" (99designs.com), which pits the designers against each other for a solitary cash prize. To further exploit the users of the site, corporations posting contests to the site can use ratings, comments and private messages to spur on the competition by helping the "designers shape their ideas to your needs" (99designs.com). That said, it has been stated multiple times previously that there are benefits beyond money that drive people to participate.
The idea of free labour, most notably argued by Terranova, is a more fitting argument against crowdsourcing than immaterial labour in some regards, mainly for the fact that it is focused on the digital economy. Free labour is "the moment where [the] knowledge consumption of culture is translated into productive activities that are pleasurably embraced and at the same time often shamelessly exploited"; free labour is "simultaneously voluntarily given and unwanted, enjoyed and exploited" (Terranova 37). This definition, applied to crowdsourcing, implies that while the participants in crowdsourced art do so out of a labour of love, their efforts are unwittingly exploited for profit. While this idea has some semblance of truth in the fact that profits may be made off the works of the crowds (as seen in the Wikipedia/Google example earlier), it is not truly exploitation in the purest sense. As Michael Strangelove argues: "if we confuse play and free expression with real exploitation, we do a disservice to those who truly suffer... when I enjoy the free music of a songbird in the forest, I am hardly exploiting the bird" (182).
Are users really being exploited, or are they merely digital songbirds? Users receive much more information from the Internet and the digital commons than they can ever give away; in this regard, the crowd is the one profiting (Barbrook). Author David Hesmondhalgh puts it best when discussing the idea of free labour: "The concept of 'free labour' is linked to some interesting ideas about power and control in cultural production in the digital era. But the frequent pairing of exploitation is unconvincing and rather incoherent..." (Hesmondhalgh 276) To thoroughly dissect the term exploitation, we need to look at it from the Marxist perspective that Terranova frames it in, not just the dictionary definition Strangelove takes advantage of. Exploitation, according to Marx is a "forced, surplus and unpaid labor, the product of which is not under the producers' control” (Ligeon 16).
Is crowdsourced labour forced? As we have seen multiple times throughout this essay, the labour is anything but forced, and in fact, it is often seen as pleasurable (Brabham, Howe 15). Members of the crowd often participate because their day jobs leave them feeling unfulfilled (Howe 29) and seek out crowdsourcing activities that are personally rewarding. These rewards can be seen as payment or compensation for the efforts of the contributors, suggesting that the labour is in fact, paid. Rewards can include solving problems, gaining new skills, and contributing to the greater good (as art does by adding to the cultural community) (Hesmondhalgh 278).
Outside of the digital realm, we see similar parallels in our communities–soup kitchen staff, soccer coaches, and numerous other volunteers–and we do not interpret their efforts as being exploited. These volunteers are compensated through rewards other than money. In the case of the soccer coach, she may be rewarded in numerous ways: team-based physical activity, teaching, winning, pride, etc. (Hesmondhalgh 277). Even in commercial art practices like iStockphoto and Threadless, "people form lasting friendships... [and] enrich everyone's experience by critiquing one another's work and teaching what they know to less experienced contributors" (Howe 14). Learning new skills and practicing old ones could be seen as a 'deferred wage' as these skills may one day help garner a higher salary or even a new career for those that participate in crowdsourced art (Hesmondhalgh 278). At the end of the day, participants may not make any money, or benefit from a deferred wage, but they also haven't invested massive amounts of time and effort in most cases. Even if they have invested large portions of their day, they did it for the pleasure of the task (DeVun).
Even Terranova concedes that "free labor, however, is not necessarily exploited labor", when she discusses the labour invested in the creation of the Internet she states "the labor of building a community was not compensated by great financial rewards (it was therefore 'free', unpaid), but it was also willingly conceded in exchange for the pleasures of communication and exchange (it was therefore 'free', pleasurable, not imposed)" (48)
The last argument for exploitation is that there is a surplus of labour, but this may not be true either, depending on how you look at it. Bradley Horowitz, vice president of the advanced development division at Yahoo! created the 1:10:89 rule which states that for every person that creates content online there are 10 people who vote/rank it, and 89 who consume it (Howe 227). This suggests that the work done in crowdsourcing is still a very low amount of creation for the population, and may not support the notion of a surplus. In reality, there are over 2.2 million Wikipedia entries (Howe 115), which would suggest that in fact there is an abundance of information to be consumed (a surplus).
Crowdsourcing, in a sense, has only been around for a few short years since Howe coined the term in 2006, but in actuality it has existed for centuries under different guises. What makes crowdsourcing unique in today's digital economy is the massive size and reach it garners through the Internet. Crowdsourcing succeeds under the principles of "faster, cheaper, smarter, easier" (Howe 71), and while cheaper may be reduced to the cost of zero in some instances, the free labour that crowdsourcing participants provide is anything but free. The true value of crowdsourcing will vary from person to person, depending on the time and energy they volunteer to the tasks offered to them, and ultimately the difference between exploitative labor and genuine pleasure is up to the individual.
Barbrook, Richard. "The Hi-Tech Gift Economy". First Monday 3.12 (2007). Web. n.d.
Berardi, Franco. The Soul At Work: From Alienation to Autonomy. Los Angeles, CA: Semiotext(e), 2009.
Brabham, Daren C. "Moving the crowd at iStockphoto: The composition of the crowd and motivations for participation in a crowdsourcing application". First Monday 13.6 (2008). Web. n.d.
Brabham, Daren C. "Moving the Crowd at Threadless." Information, Communication & Society 13.8 (December, 2010): 1122-1145. CrossRef. Web. n.d.
Browne, Sarah. "Crowd Theory Lite 'The Crowd' in Participatory Art and Pop Economics". Circa. 126 (2008): 33-39. Web. 14 December 2013.
Chimero, Frank. "Improvisation and Limitations" The Shape of Design (2013). Web. n.d.
Cuenca, "Alberto López. Artistic Labour, Enclosure and the New Economy". Afterall: A Journal of Art, Context, and Enquiry. 30 (2012): 4-13. Web. n.d.
DeVun, Leah. "(Q&A) Your Assignment: Art". Wired. Wired.com, 9 July 2007. Web. n.d.
FAQ. Amazon Mechanical Turk. Amazon. (n.d.). Web. 16 December 2013.
FAQ About Spec Work. No!Spec. nospec.com (n.d.) Web. 15 December 2013.
Grover, Andrea. "Phantom Captain: Art and Crowdsourcing" Apex Art. apexart.org, 2006. Web. n.d.
Hardt, Michael., and Antonio Negri. Empire. Cambridge, Mass.: Harvard University Press, 2000.
Hesmondhalgh, David. "User-generated Content, Free Labour and the Cultural Industries". ephemera 10.3/4 (2010): 267-284. Web. n.d.
Howe, Jeff. Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business. New York: Crown Business, 2008.
How It Works. 99 Designs. 99 Designs. (n.d). Web. 16 December 2013.
How it Works. Threadless. threadless.com (n.d.) Web. 16 December 2013.
Lazarrato, M. "Immaterial Labour.(P. Colilli & E. Emery, Trans.)." Radical Thought in Italy: a Potential Politics (1996): 133-147.
Ligeon, Luciano. "Crowdsourcing: Labour Struggles Revisited". University of Utrecht, 2012. Web. n.d.
Literat, Ioana. "The Work of Art in the Age of Mediated Participation: Crowdsourced Art and Collective Creativity". International Journal of Communication 6 (2012): 2962–2984. Web. n.d.
Ross, Andrew. "In Search of the Lost Paycheck". Digital Labor: The Internet as Playground and Factory. Ed. Trebor Scholz. New York: Routledge, 2013. 13-32. Print.
Tim Burton's Cadavre Exquis. Toronto International Film Festival. burtonstory.com (n.d.) Web. 16 December 2013.
Terranova, Tiziana. "Free Labor". Digital Labor: The Internet as Playground and Factory. Ed. Trebor Scholz. New York: Routledge, 2013. 13-32. Print.
Tryon, Chuck. On-demand Culture: Digital Delivery and the Future of Movies. New Brunswick, N.J.: Rutgers University Press, 2013.
Scholz, Trebor. "Crowdsourcing and Participation: Cheaper by the Dozen an Introduction to 'Crowdsourcing'". Net Works: Case Studies in Web Art and Design. Ed. xtine burrough. New York: Routledge, 2013. Web.
Star Wars Uncut - The Fan-made Galactic Saga continues. Star Wars Uncut. Casey Pugh. starwarsuncut.com (n.d.) Web. 16 December 2013.
Strangelove, Michael. Watching YouTube: Extraordinary Videos By Ordinary People. Toronto: University of Toronto Press, 2010.
Swarmsketch. Swarmsketch. Peter Edmunds. swarmsketch.com (n.d.). Web. 16 December 2013.