Paul Resnick's Occasional Musings
Paul Resnick

[ website | Paul Resnick's homepage ]
[ userinfo | livejournal userinfo ]

Howard Dean, Social Capital, and the Internet: Global Organizing of Local Activity [08 Jul 2003|05:43pm]
For years now, I've been monitoring social capital building efforts on the Internet. I think I've now accumulated enough examples to articulate an important category of social capital building opportunities on the Internet: non-local sites that help organize local activities.

General architecture:

  1. there's some interest-based (not geographic) draw that brings lots of people to a web site.

  2. people type in their zip codes, and get matched to local activities (on-line or especially off-line) where they connect with other people geographically near them.



Examples:

  1. upmystreet.com conversations in England (conversation system where you see messages from people geographically near you)

  2. geocaching.com GPS-based "treasure hunt" game (though interaction may be limited to hiding and finding treasures)

  3. Net Day (use map to pick a school; get connected to others who picked the same school to wire).

  4. bettertogether.org (though we never quite figured out how to get people involved, and then connected to each other, after they were matched to stories in their geography)

  5. Now the Dean for President campaign. There's a buzz now that's driving people to their site. But it's interesting what happens when you get there. Instead of just signing up and waiting for the campaign to contact you, you can create a local event, and others can then sign up for them. In both Michigan and Pennsylvania, I found a fair bit of activity that was at the pre-organizational stage. My wife Caroline went to the site intending just to make a financial contribution and ended up signing up to go to a house party.
    In other words, Dean supporters are using the Internet to find each other and then create local organizations on their own.

CTCs and eBay [07 Jul 2003|12:04pm]
As I left the eBay Live conference (see entry below), on my way back to the CTCTNet conference (I'm on their advisory board), I had an epiphany that eBay could be a good platform for creating small businesses in low-income communities, and that Community Technology Centers could be a good venue for helping such businesses to form. Selling in volume on eBay would force budding entrepreneurs to learn about overhead, inventory management, customer service, etc., not to mention digital photography and effective on-line merchandising.

The ability to sell things to a larger market beyond one's own community has a number of important advantages, including bringing in income from outside and broadening the horizons of residents. Without the eBay/UPS/FedeX platform, however, the barriers to entry were pretty high. Of course, you still have to have something to sell, but I think there are a couple of possibilities:

  1. Run a consignment shop. At eBayLive's expo there was a company called dropitoff.com that was trying to franchise drop off stations. But this could be done independently. This would provide some healthy competition for the local pawn shops, and it might be possible to collect "donations" from wealthier surrounding neighborhoods.

  2. Run a retail resale business, buying wholesale lots from eBay, breaking up those lots and selling them individually. The profit margin on this would be modest, probably about the cost of labor if the business was run efficiently. But that's not a bad business model, either. They'd have to be careful about not buying inventory that has a short shelf life, though (electronics or faddish items). eBay even has categories for wholesale lots, and there are guides to wholesale buying for resale on eBay.


I relayed the idea to John Zoltner (CTCNet development staff) and Kavita Singh (CTCNet executive director). John said that he'd had contact with a national group interested in providing entrepreneurship education through CTCs but that the concern has always been that there aren't that many real entrepreneurial opportunities, especially for young people who are still in school, so the entrepreneurship education would end up being abstract rather than practical. Perhaps eBay is the missing piece of the puzzle.

eBay Live trip report [02 Jul 2003|03:51pm]

I dropped in on eBay Live in Orlando for 24 hours. This was the second annual convention that eBay has put on. At the April workshop on reputation systems we held at MIT, Peter Kollock from UCLA, as well as Brian Burke and Amjad Hanif from eBay had encouraged everyone to attend if they wanted to get a feel for the culture and the passions of eBay members, especially power sellers.

Feedback Session

I timed my visit around the session on the feedback forum that Brian organized. Amjad reminded the audience of the recent move to display percentage of positive feedback, and presented some of the changes in preparation, which are mostly user interface changes to the “scorecard” page.

Most of the session was Q&A from the floor. All the questioners were high volume sellers (of the first 20, only one or two had scores below 1000). The number one concern was “unfair” negative feedback, usually from new members who are unfamiliar with the norms of working out problems and leaving negative feedback only for unresolved problems, and from non-paying bidders who retaliate after receiving negative feedback. EBay is unwilling to make any editorial judgments about feedback, lest it be held legally responsible for all the feedback content. One of the recent changes they’ve made is to add a “think about it and confirm that you mean it” screen between submitting negative feedback and having it posted.

I think there are two problems with the official and community encouragement to resolve disputes before leaving negative feedback. First, patterns of mild dissatisfaction are not recorded, so lots of useful information is lost. Second, sellers have become overly sensitive any negative or even neutral effect because it is so rare. If negative feedback were given 5% or 10% of the time, on average, then sellers would worry about keeping their percentage down, but wouldn’t be as concerned about any particular feedback.

In the past founder Pierre Omidyar has urged the community to give more negative feedback and I agree.

There was also discussion about making the feedback scorecard better reflect the total number of transactions and/or repeat business, rather than just the number of distinct transaction partners who had left feedback. In the analyses I have been doing with our data set from 1999, I have switched over to treating trading partnerships as the basic unit of analysis rather than treating feedbacks (or transactions) as the basic unit of analysis. With my approach, eBay might show that a seller had 300 partners; 160 of them gave positive feedback at least once and 2 gave negative; 83 of them were repeat customers; the average number of transactions per partner was 1.5, for a total of 450 items sold. Later this summer, I’ll be analyzing my 1999 data to see whether percent of partners that gave any feedback is correlated with percent of feedback that is negative (I suspect it is), so that percent of partners with no feedback might be a good measure of the not-thrilled customers that’s more reliable even for small numbers; it’s hard to get reliable estimates of low-prevalence events like really negative experiences.

In the slightly longer term, Jeff Taylor from eBay said they are considering displaying the percentage of positive feedback from a recent time window rather than or in addition to the percentage for the entire history. Several research papers recently have pointed out that if the entire history is used, as a seller accumulates a large number of feedback the marginal impact of an additional positive or negative feedback goes down (it’s impact on the percent negative, or on Bayesian updating, goes down), and thus the incentive effects are reduced (Dellarocas, Wood). Of course, judging from what I saw at this conference, the incentive to avoid negative feedback is mostly a psychological one for sellers, not based on its impact on sales, and may be quite strong in any case. Thus, the move to display a percent positive based on more recent history seems like a good one. I would suggest, however, that it be based not a time window (last six months) but on a transaction window (last 200 trades, or partners, or feedbacks, depending on what the unit of analysis is).

Reputation Research Opportunities

Sellers seemed to think that the problem of clueless new buyers is worse today than it used to be. It would be interesting to do a comparison over time to see if newcomers are more likely to leave negative feedback than they were a few years ago. I talked to eBay’s data analysis guru (but whose name I didn’t catch) whose hypothesis was that it wouldn’t be different.

While wandering the convention floor, I came across a couple of potential research opportunities. Square Trade offers a “seal of approval” to sellers who get their identity and credit checked and commit to using Square Trade’s dispute resolution procedures. They have data on which they’ve done preliminary analysis comparing sellers who’ve signed up to see if they get higher prices or probability of sale after signing up. It would be interesting to subject that data to rigorous analysis.

A company called Andale (pronounced Ahn-da-lay) was one of seemingly dozens of providers of tools for high volume sellers. In addition to auction listing services, they have a “Research” product that tells sellers things like how they’re doing compared to other sellers in the category, and, using historical data, what’s the best day and time to have an auction end. They also have counters for tracking how often pages are viewed and calculated “sell-through” rates. It would be interesting for feedback researchers to see if better reputation is associated with higher sell-through rates. As with other measures of buyer response to reputation such as bids and sales, there are many possible confounds to contend with.

EBay’s Vision: global, level playing field

I went to a general Q&A session with Meg Whitman and other top execs. This was partly a love fest. One comment from the floor was “we’d all like to give you 10,000 positive feedback.” Whitman answered a question about “how do you avoid burnout?” by saying the eBay staff got their energy from seeing all the members who were using the service. (This seemed to be somewhat true, incidentially. Though some eBay staff I had dinner with were a little weary after a day of interactions with members/fans, they did seem to have a sense that their jobs were meaningful, moreso than their previous jobs working at the likes of 3Com or Apple.) Apparently, I missed the real love fest on the first morning. They flew in Weird Al Yankovich in the corporate jet between his Thursday and Friday night concerts in the Midwest to sing his song about eBay.

I heard two interesting comments from Whitman in this session. First, their direction is to go global. They already have 11% of their transactions crossing international borders (so she said; sounds implausible to me) and they want to try to enable anyone to do reduce the barriers. Second, they want to make sure to keep a level playing field for big and small sellers. She said that their past special deals with Disney, for example, were probably a mistake in hindsight and they wouldn’t do it again.

I missed founder Pierre Omdiyar’s appearance. I’m told he reiterated his view that, “Most people are basically good”. This is, of course, contrary to economists’ assumption that people are basically self-serving. Perhaps it goes partway toward explaining what Richard Zeckhauser and I have called Yhprum’s Law, that systems which shouldn’t work sometimes do.

1. Introduction [22 May 2003|11:22am]
Online Reputation Mechanisms
A Roadmap for Future Research

Summary Report of the
First Interdisciplinary Symposium on Online Reputation Mechanisms,
April 26-27, 2003, Cambridge, MA
Draft Versión 1, May 21, 2003

Chrysanthos Dellarocas, MIT (dell@mit.edu)
Paul Resnick, U. Michigan (presnick@umich.edu)

Online reputation mechanisms are large-scale online word-of-mouth communities in which individuals share opinions on a wide range of topics, including companies, products, services, and even world events (Dellarocas, 2003). Best known so far as a technology for building trust and fostering cooperation in online trading communities, such as eBay, these mechanisms are poised to have a much wider influence on consumer behavior and public opinion formation that will impact both online and offline organizations.

The rising practical importance of online reputation systems invites rigorous research in this largely virgin territory. Do these systems truly promote socially desirable outcomes? To what extent can they be manipulated by strategic buyers and sellers? What is the best way to design them? How should buyers (and sellers) use the information provided by such mechanisms in their decision-making process? This is just a small subset of unanswered questions that invite exciting and valuable research.

In order to answer these questions, collaboration is needed between several traditionally distinct disciplines, such as economics, computer science, marketing, law, sociology and psychology. In each of those communities, researchers are actively working on aspects of reputation systems and their work has been well received within their own disciplines.

An informal community had been forming around the topic. A few researchers had met each other at disciplinary conferences and workshops. There is at least one web site (http://databases.si.umich.edu/reputations) devoted to building the community of research on reputation mechanisms. At the same time, a number of established and startup companies, such as Epinions, BizRate and OpenRatings, have been basing their business models on the accumulation and dissemination of reputation information. So far, however, most of these people had not had a chance to meet their colleagues to identify opportunities for cross-fertilization of ideas.

The First Interdisciplinary Symposium on Online Reputation Mechanisms took place at MIT, on April 26-27, 2003 with the aim of bringing together the leading researchers from various disciplines that are active in the area of reputation systems, together with a few people from industry who are responsible for practical implementations of reputations systems. The objective of this symposium was to help us better understand how these various lines of work connect to one another and how, together, they can contribute to the design and implementation of better feedback mechanisms for electronic commerce and our digital society at large.

Participants Reference Discipline

Economics/IT and Economics 23
Computer Science/AI 13
Sociology/Psychology 10
Industry 8
Total 54

Participants Affiliation

Academia 40
Industrial Research 4
Industry 8
Government 2
Total 54

Doctoral Students 18



Table 1. Composition of Symposium Participants.

Funding for this symposium was provided through grants from the National Science Foundation (Award number 0209136, CISE/Digital Society & Technologies) and the Center for eBusiness @ MIT. Table 1 summarizes the composition of the symposium participants.

The rest of this report summarizes the principal findings of this symposium. Section 2 outlines why we believe that this field is going to play an increasingly important role in business and society and therefore merits rigorous interdisciplinary study. Section 3 summarizes the principal themes of the work presented in the symposium. Collectively, these represent a snapshot of the state-of-the-art in this field today. Section 4 discusses important open research questions identified during the symposium. Finally, Section 5 concludes.

2. Why is This Field Important? [22 May 2003|11:19am]
Reputation mechanisms have initially attracted attention as a mechanism for building trust and fostering cooperation in online trading communities, such as eBay, where more traditional quality assurance mechanisms (commercial law, government regulation, etc.) do not work as well. They have also emerged as an attractive approach for inducing "law and order" in loosely coupled networks of computer systems, such as software agent communities and peer-to-peer networks.

We believe that feedback mechanisms are poised to have a much wider impact on consumer behavior and public opinion formation. Their growing popularity can, therefore, have potentially important repercussions for a variety of business and public policy institutions ranging from marketing and advertising to litigation and the political process.

The following recent events indicate the growing influence of online feedback in business and politics.

  • As early as 1994, the negative publicity generated through online feedback forums regarding an obscure and rather insignificant defect of the first generation of Pentium processors, led to hundreds of reports on the mass media, bad jokes on late-night television, and a $475 million fourth-quarter debit on Intel's balance sheet (Coe et al., 1995).
  • In December 2002, the vigorous denunciation of Senator Trent Lott's controversial remarks about Strom Thurmond in online "weblogs" forced the mainstream media (who, remarkably, had made a conscious or unconscious decision to not give publicity to the incident) to take notice and eventually led to Senator Lott's resignation from his position as majority leader (Economist, 2002).
  • In February 2003, Jeff Bezos, the CEO of Amazon.com decided to cancel all plans for any television or general-purpose print advertising because he believes that his company is better served through word-of-mouth generated through the Internet (Hansell, 2003).

The above examples indicate that online feedback forums can exert powerful influence in an increasing number of social and business activities. They, therefore, represent an emerging social force that needs to be carefully studied, understood, and harnessed for society's good.
The majority of existing online feedback mechanisms have been developed by a variety of commercial and community groups and based on ad hoc principles and ideas. As these powerful systems proliferate, it is important to conduct rigorous research in order to understand the full potential benefit of these mechanisms for business and society as well as to protect society from potentially harmful consequences of poorly designed feedback mechanisms. Some important questions that such research should attempt to answer include:

  • What is the net impact of these mechanisms in economic efficiency?
  • What is the net impact of these mechanisms in social fairness?
  • Who benefits and who loses from the introduction of a feedback mechanism?
  • Are there domains in which the creation of such mechanisms is particularly beneficial? Are there domains where such mechanisms can be harmful?
  • To what extent do the design details of the mechanism impact the resulting social outcomes?
  • What are the risks related to abuse of such mechanisms?
  • How can such risks be avoided?
  • How should organizations measure and react to online feedback?

In order to answer the above questions, a sustained research effort is needed. This effort requires collaboration among several disciplines, including:

  • Economics. Economists have studied reputation formation and social learning phenomena (in offline settings) for at least twenty years. This body of work is a useful foundation for understanding how these phenomena extend in the online domain.
  • Computer Science/MIS. Online reputations are computed by computer systems that aggregate feedback provided by online community members as well as information implicitly mined from the Web and other public information sources. Some otherwise desirable techniques for aggregating and filtering information may not scale well to large amounts of information. In addition, in many cases it will be desirable to perform calculations in a distributed manner, perhaps in the presence of some system elements that can not be trusted to perform computations as expected. Computer Science/MIS researchers bring experience in the design of scalable, distributed, and secure computation.
  • Sociology and Psychology. It is well-known that human behavior does not conform to the assumptions of full rationality that are pervasive in economic modeling. Psychologists can explain and predict behavior of people in the presence of reputation systems based on emotional as well as cognitive factors. Sociologists can provide insights into the institutional and societal level effects of reputation systems.
  • Management Science. Management scientists will play an important role in understanding the impact of online reputation on a variety of management issues including brand building, marketing, product development and quality assurance.
  • Political Science. Given the growing potential of online reputation mechanisms to influence public opinion, and emerging applications in the structuring of on-line deliberation, the attention of political scientists will be very valuable in understanding the implications of such systems in the political process.

Some private entities will likely benefit directly from research on these issues. Thus, corporations such as eBay should be encouraged to begin funding some of the research that will directly affect their businesses. However, much of the required research is of a more fundamental nature that will benefit many businesses, non-profit organizations and government agencies. It will also help society understand and cope with the broader social and political implications of this phenomenon. Thus, like much scientific research, research on reputation systems is a public good that will be best funded by government.

3. A Snapshot of the State-Of-The-Art [22 May 2003|11:17am]
The papers presented in the symposium collectively represent a snapshot of the state-of-the-art in the field. Current research on reputation mechanisms concentrates around three major themes:

  • Analyzing the performance of existing reputation mechanisms

  • Proposing novel designs for reputation mechanisms and modeling their impact

  • Exploring the use of reputation mechanisms in novel settings

The following sections provide the main themes of work presented in the symposium.

3.1 Theme 1: Analyzing the performance of existing reputation mechanisms [22 May 2003|11:16am]
[Error: Irreparable invalid markup ('<no [...] section,>') in entry. Owner must fix manually. Raw contents below.]

<no text in this section, just the subsections below>

3.1.1 Field studies [22 May 2003|11:14am]
A substantial amount of recent research has focused on empirical evaluations of existing reputation mechanisms [2, 3, 4, 5, 6]. The majority of these works focus on eBay's feedback mechanism and concentrates on studying the relationship between a seller's reputation score, auction closing prices, and the probability that an auction will receive at least one bid. Researchers have conducted observational studies of listings in particular categories, with increasing methodological sophistication. One controlled experiment sold matched items under different seller identities with different feedback profiles. In general, researchers have found that buyers do reward sellers with better reputations, but only by a small amount. Results have not been entirely consistent, however, in particular about whether positive feedback and/or negative feedback makes a difference. A panel compared some of the methods and results, and a summary table was distributed.

Other topics addressed by field studies include understanding the drivers and evolution of buyer participation on eBay's feedback mechanism. Preliminary results indicate that that a buyer's propensity to leave feedback for a seller has a positive correlation with the amount of positive seller reciprocation that this buyer has experienced in the past. Furthermore, experienced buyers (buyers who have completed large numbers of transactions), as well as buyers who transact with high frequency tend to leave feedback less often.

The industry representatives from eBay and epinions have conducted customer focus groups and observed their users' reactions to various proposed and actual changes in their systems [1]. They emphasized two critical factors. The first is that people care a lot about their own reputation scores, even beyond any commercial impact those scores might carry. Second, their users strongly prefer metrics that are easily understood. For example, if there are several separate scores that are then aggregated into a composite score, they want to understand how the components are weighted.

3.1.2 Lab Studies [22 May 2003|11:11am]
An increasing number of experimental studies also aim to shed light on how people react to reputation mechanisms [27, 28, 29, 30, 33]. Again, most studies relate to systems that are similar to eBay. Important themes in experimental studies include:

  • The relative importance of positive vs. negative reputation
  • The power of reputation relative to that of stable partnerships. In theory, under the assumption of perfect public monitoring of transaction outcomes, reputational incentives ought to lead to similar levels of cooperation as stable partnerships. Lab studies have tested this assumption in practice [27, 33]. The initial results are conflicting. One study [27] found that subjects tend to cooperate less in environments where traders change partners, but observe each partner's past history than in environments where stable partnerships are formed. Another study [33] did not find significant difference in cooperation levels in the two settings.
  • The dual role of reputation as sanctioning and signaling. An accumulated reputation acts as a signal to buyers about whether the seller is likely to be trustworthy in the current round. If sellers are not strategic, it provides information about their underlying quality level. Even if they are acting strategically (providing better goods or service when the incentives warrant it), the past reputation may separate sellers who have an incentive to act better or worse in the current round. In these cases, feedback is also acting as a sanctioning mechanism: the seller's expectation that the future value of receiving feedback in this round is sufficient to incent good behavior in this round. Thus, depending on whether sellers have a fixed underlying quality or whether they vary their quality depending on incentives, reputation acts either as signal only or as both signal and sanction.
    Lab studies can determine the extent to which buyers believe that feedback works through sanctioning or only through signaling. When it works only through signaling, the signals are more precise the more feedback that has accumulated. When it works through sanctioning as well, however, a seller should be more attentive to the feedback they are about to receive when they have not yet accumulated much. Through experiments, researchers can observe whether buyers attribute greater trustworthiness to sellers who have a lot of feedback, but may not care much about the next one they receive, or whether they attribute greater trustworthiness to sellers who do not yet have a lot of feedback and thus should care more about their feedback from the current transaction.

3.1.3 Analytic studies [22 May 2003|11:10am]
A game-theoretic model of eBay's mechanism was presented in the symposium [32]. The analysis concluded that eBay's mechanism is capable of inducing high (but not perfect) levels of cooperation and that the unusually low fraction of negative feedback on eBay is a natural consequence of equilibrium behavior. Furthermore, it investigated the impact of incomplete feedback submission and easy name changes and concluded that (i) eBay's mechanism can be more efficient in the presence of incomplete feedback submission if it treats missing feedback as positive feedback and (ii) one approach for discouraging players from changing their identities is to start new players with a profile that corresponds to "bad reputation" and have them "pay their dues" at the initial phase of the game. An earlier paper, not presented at the conference, showed that no other approach can be significantly better even when there is complete submission of feedback: distrust of newcomers through a "paying your dues" equilibrium is an inherent social cost of systems with easy name changes (Friedman and Resnick, 2001).

3.2 Theme 2: Proposing novel designs for reputation mechanisms [22 May 2003|11:10am]
Most of the work in this theme comes from researchers in computer science and multi-agent systems. The general consensus was that there are some very interesting ideas that need to be more rigorously validated by analyzing them in an economic framework. This is an area where there are great opportunities for collaboration between computer scientists and economists, even though researchers from both fields acknowledged the existence of a "language barrier" (which, we are confident, can be overcome through more sustained contact between researchers of the two disciplines.)

Mechanisms for eliciting truthful feedback. In most online systems feedback submission is voluntary. In the absence of concrete incentives, online community members may thus refrain from providing feedback or provide intentionally or unintentionally untruthful feedback. A number of researchers are working towards developing mechanisms that provide strict incentives to online community members to both participate (i.e. provide feedback) as well as truthfully report their observations [15, 31].

Implicit extraction of reputation. An important theme of this workshop was the use of data mining techniques that can automatically extract reputational information from publicly available networked data structures, such as the Web, Usenet groups, etc [7, 13, 26]. An impressive amount of information about someone's social standing, past behavior, and interaction habits can be inferred in this way. Such implicit reputation mechanisms are an intriguing complement to mechanisms that rely on explicit feedback. They can be particularly useful in terms of bootstrapping feedback mechanisms (i.e. substitute for feedback during the initial phase when feedback is scarce) as well as in situations where feedback is unreliable or subject to strategic manipulation.

Distributed feedback mechanisms. Most commercial feedback mechanisms are based on centralized architectures. That is, feedback is solicited, and stored in a single repository, controlled by a single organization (e.g. eBay, Epinions, Amazon). Motivated by issues of privacy, trust, and scalability, some researchers are beginning to look at distributed feedback mechanism architectures [8, 13]. In such systems, agents receive reputational information from a variety of sources, including direct experience, feedback from third parties, and implicitly extracted information. An important challenge is to develop algorithms through which these sources of information are combined in a "sensible" way to adjust the agent's beliefs. A further challenge relates to modeling the effectiveness of such complex systems and comparing it to that of centralized feedback systems. Finally, these systems may need to be resilient to the presence of "strategic" agents who attempt to influence the calculations for their own benefit or even "malicious" agents who merely attempt to render the system ineffective.

3.3 Theme 3: Exploring the use of reputation mechanisms in novel settings [22 May 2003|11:09am]
The majority of current research on reputation mechanisms has focused on the application of such mechanisms in online marketplaces. Increasingly, researchers are looking into other domains where such systems can play a useful role. Some examples include:

· Peer-to-peer networks [9]

· Wireless grids [18]

· Professional guilds [23]


In contrast to the above proposals, Jeff Ely presented a paper where he outlined an (abstract) setting in which publicly available reputational information ends up harming otherwise honest service providers [11]. His paper shows that the development of reputation mechanisms may not be socially beneficial in all cases.

4. The Way Forward [22 May 2003|11:08am]
The closing session of the symposium consisted of a brainstorming session during which participants expressed their views concerning the most important open questions that the field should address in the near future. The following is a summary of the main points made.

4.1 Reputation Formation in Competitive Settings [22 May 2003|11:07am]
Most analytical and experimental studies of reputation formation in trading environments have made the assumption that the seller is a monopolist. The dynamics of reputation apparently change in interesting ways when there are several competing sellers. Heski Bar-Isaac presented some preliminary work on this topic [20]. His most striking finding was that competition changes reputational incentives in a non-monotonic manner: the power of reputation rises, then falls, then rises again as a market becomes more competitive. This is an interesting area where more analytical work in needed.

4.2 Usability Considerations [22 May 2003|11:06am]
Industry participants ranked usability of reputation mechanisms near the top of their open issues. Users of eBay and Epinions are currently having difficulties understanding what the reputation measures mean and, more importantly, how to factor them into their decision-making. A lot of the analytical models presented by researchers in this symposium are substantially more complex than eBay's current mechanism. An important research and practical question therefore revolves around the design of mechanisms that are both theoretically sound as well as easy to understand for the average Internet user.

4.3 Representations and Ontologies for Reputation [22 May 2003|11:05am]
There is a lack of consensus regarding the fundamental concepts that relate to reputation mechanisms. The current diversity of designs and concepts associated with reputation mechanisms might be contributing to the confusion faced by consumers when using such systems. The development of ontologies in this domain would be a step forward, both for developing more standardized and usable reputation mechanisms, as well as for developing machine-readable reputation measures that could be used in the context of peer-to-peer networks and multi-agent systems.

4.4 Relationship between Online and Offline Reputation [22 May 2003|11:04am]
Online reputation mechanisms are increasingly aggregating opinions about offline firms and products. For many of these firms, consumers have access to offline measures of reputation based on brands, physical stores, advertising, etc. As more and more consumers visit online forums it is important to understand how a firm's online and offline reputations interact and how they influence consumer purchase decisions [24].

4.5 When is Reputation Bad [22 May 2003|11:03am]
The paper presented by Jeff Ely [11] discussed an interesting setting where the availability of public reputational information can be harmful for an (otherwise honest) service provider. It is important to understand the range of practical settings in which these results might apply, and therefore to identify classes of settings for which the development of online feedback mechanisms may not be a good idea.

4.6 Emotional Factors [22 May 2003|11:02am]
Practitioners and sociologists seemed to agree that people associate an emotional value to their online reputation profile. eBay members are proud to be "power sellers" (a distinction earned once their reputation score surpasses a certain level) and their anger over receiving negative feedback is often disproportionate to its actual negative impacts on their businesses. The emotional factors associated with reputation mechanisms thus seem to be an important driver of these mechanisms' effectiveness and deserve further study.

4.7 Social impact of ubiquitous reputational information [22 May 2003|11:01am]
The combination of online feedback mechanisms and implicit reputation extraction systems discussed in the symposium points to a future where lots of reputational information about firms and individuals will be readily available on the Internet. Some symposium participants referred to this trend as the "end of practical obscurity" and suggested that its social consequences need to be carefully considered.

4.8 Social impact of centralized reputation repositories [22 May 2003|11:00am]
The current generation of online feedback mechanisms relies on a centralized repository of ratings for online community members. As the importance of such information grows, the privacy and market power implications of having so much sensitive information under the control of a single organization (the mechanism operator) must be carefully understood.

4.9 Multi-cultural aspects of reputation mechanisms [22 May 2003|10:59am]
The majority of empirical and experimental studies of online reputation mechanisms have been performed in the U.S. It is important to understand how cultural differences may affect the way in which such mechanisms influence behavior in other cultures.

4.10 Impact of reputation mechanisms on traditional institutions [22 May 2003|10:57am]
Several participants remarked that current research on reputation mechanisms is rather narrowly focused on the study of existing systems, most notably eBay's system. As we argued in Section 2, online feedback has an increasingly important role to play in a variety of other settings that involve offline decision making, such as brand building, product development, and public opinion formation. More macro-level research is needed in order to understand how the spread of online feedback mechanisms might impact more traditional institutions such as advertising, the media, and the legal system, in the context of the above settings.

4.11 Data Sets [22 May 2003|10:55am]
Many researchers are interested in analyzing data, especially from eBay. While eBay staff would like in principle to help move research in this area forward, they have limited internal capacity to negotiate terms of data sharing agreements and to perform custom extractions of data. They are investigating ways to prepare standard data sets and a standard confidentiality agreement that would be available to researchers.

4.12 Future Publications [22 May 2003|10:50am]
There would be some clear benefits to having some kind of edited collection representing the current state of the art of current research. Some participants argued in favor of a book with full length articles. Others preferred a venue such as Communications of the ACM, which has wide circulation but publishes only very short articles (3000 words or less). Finally, there was a suggestion of getting journals in different fields to do parallel special issues. We are still thinking this through, in light of both what would be best for the field and what work we're willing to do. In the meantime, everyone is encouraged to post working papers or published papers (your own or others) at the Reputations Research Network web site, http://databases.si.umich.edu/reputations/.

4.13 Future Meetings [22 May 2003|10:45am]
There was widespread sentiment that this should not be the last get-together of this research network. Since we come from different fields, there is no natural choice of a larger meeting to which we might attach a workshop. Thus, it would probably have to be a stand-alone meeting. Attracting NSF funding will probably get more difficult as we get more established, so we may need to think about a meeting where most participants pay their own way. Another issue is how formal to make the meeting. This one was quite informal in terms of deadlines, selection processes, and to some extent the format of the program itself. There will be a natural tendency to become more formal, and it is not clear whether that tendency should be resisted. Michael Macy expressed some interest in trying to convene a meeting next year, or perhaps others would be interested as well. The organizers of this year's meeting would be happy to serve in some role, but would like to see others step forward to take primary leadership.

5. Concluding Remarks [22 May 2003|10:40am]
The First Interdisciplinary Symposium on Online Reputation Mechanisms was a success at bringing together researchers from diverse disciplines. The meeting pointed the way towards much work that needs to be done. It also highlighted the differences in vocabulary and assumptions among disciplines, and differences in approaches and time scales between researchers and practitioners.

There is much work to be done. But it is important that research be conducted now, in the formative phases of this technology and the social practices surrounding it. There are likely to be path-dependent effects in the deployment and use of online reputation systems, so it is important that researchers develop insights into the functioning and impacts of online reputation systems while they can still have a larege impact on practice.

6. References [22 May 2003|10:04am]
· Coe, T., et al. (1995) Computational aspects of the Pentium affair. IEEE Computational Science and Engineering 2 (Spring 1995), pp. 18-30.
· Dellarocas, C. (2003) The Digitization of Word-of-Mouth: Promise and Challenges of Online Feedback Mechanisms. Management Science (forthcoming).
· Economist (2002) Missisippi Burning. The Economist, December 21, 2002, p. 25.
· Friedman, E. and Resnick, P. (2001). The Social Cost of Cheap Pseudonyms. Journal of Economics and Management Strategy 10 (1), pp. 173-199.
· Hansell, S. (2003) Amazon decides to go for a powerful form of advertising: lower prices and word of mouth. The New York Times, February 10, 2003, page C8.

7. APPENDIX: Symposium Program [22 May 2003|09:00am]
Available online (with links to papers and slides) at http://www.si.umich.edu/~presnick/reputation/symposium/agenda.htm
Saturday April 26, 2003
8:30-9:00 Registration
9:00-9:30 Introduction and Overview
9:30-11:00 Industry Panel
[1] Session Chair: Peter Kollock (UCLA )
11:00-11:15 Break
11:15-12:15 Panel: Field Studies on Impact of Reputations
Session Chairs: David Reiley (U. of Arizona) and Paul Resnick (U. Michigan )
Panelists:
[2] · David Eaton (Murray State U. )
[3] · Jeffrey Livingston (U. Maryland )
[4] · Mikhail Melnik (Georgia State University )
[5] · Pai-Ling Yin (Stanford )
[6] · Sulin Ba (U. Connecticut)
12:15-13:30 Lunch
13:30-15:00 Distributed Reputation Mechanisms
Session Chair: Roger Dingledine (The Free Haven Project)
Papers:
[7] · Extracting Reputation in Multi Agent Systems by Means of Social Network Topology. Josep M. Pujol, Ramon Sanguesa and Jordi Delgado (Technical U. of Catalonia)
[8] · Referrals for Reputation. Munindar P. Singh (North Carolina State U.)
[9] · Trust and Reputation in P2P Networks. Zoran Despotovic and Karl Aberer (EPFL Switzerland)
15:00-15:15 Break
15:15-16:45 Theory and Simulation
Session Chair: Steve Tadelis (Stanford)
Papers:
[10] · Is Bigger Better? Customer Base Expansion through Word of Mouth Reputation. Arthur Fishman (Bar Ilan U.) and Rafael Rob (U. of Pennsylvania)
[11] · When is Reputation Bad? Jeffrey Ely (Northwestern U.) Drew Fudenberg (Harvard) David Levine (UCLA)
[12] · The Dynamics of Reputation. Fang Wu (Stanford U.)
16:45-17:00 Break
17:00-18:00 Boasters Session I
[13] · The ReGreT system. Jordi Sabater (IIIA-CSIC) and Carles Sierra (IIIA-CSIC)
[14] · Reputation in Anonymity Systems. Roger Dingledine (The Free Haven Project)
[15] · Incentive Compatible Reputation Mechanisms. Radu Jurca (EPFL) and Boi Faltings (EPFL).
[16] · A Reputation Systems Evaluation Framework. Audun Josang (DSTC) Shane Hird (DSTC) and Eric Faccer (DSTC)
[17] · Employing limited-trust chains to limit the effect of manipulation of reputations in online communities. Vineet Kumar (UCSD)
[18] · The role of reputation in virtual markets for wireless grids. James Howison (Syracuse U.)
18:00 Dinner, MIT Faculty Club

Sunday April 27, 2003
9:00-10:00 Boasters Session II
[19] · Decomposing Reputation Effects: Sanctioning or Signalling? Ko Kuwabara (Cornell Univeristy)
[20] · Something to prove: uncertainty and reputation. Heski Bar-Isaac (Northwestern U.)
[21] · Buyer Fatigue: How Seller Reciprocation and Experience Impact Buyers' Participation in Reputation Systems. Chrysanthos Dellarocas (MIT) Ming Fan (U. Washington) and Chuck Wood (U. of Notre Dame)
[22] · A mental model approach for understanding how reputation systems work. Viswanath Avasarala (Penn State U.)
[23] · Reputation in Online Professional Guilds. Cynthia Typaldos (Working Persona | Software Product Marketing eGroup)
[24] · Off-line and On-line Reputation in a New Car Market: Demand for cars on eBay. Christopher Adams (Federal Trade Commision)
10:00-10:30 Research prototypes
[25] · PACT -- A reputation based message board. Eric Friedman (Cornell U.)
[26] · Social accounting measures of conversational social cyberspaces. Marc A. Smith (Microsoft Research)
10:30-10:45 Break
10:45-12:15 Panel: Lab Experiments
Session Chair: Michael Macy, Cornell U.
Panelists:
[27] · Axel Ockenfels (Max-Planck Institute)
[28] · Martin Ekstrom (Stanford U. )
[29] · Dan Houser (George Mason U.)
[30] · Toshio Yamagishi (Hokkaido U. )
12:15-13:15 Lunch
13:15-14:45 Eliciting Ratings and Computing Reputations
Session Chair: Drazen Prelec (MIT )
Papers:
[31] · Eliciting Honest Feedback in Electronic Markets. Nolan Miller (Harvard) Paul Resnick (Univ. of Michigan) and Richard Zeckhauser (Harvard)
[32] · Efficiency and Robustness of Binary Feedback Mechanisms in Trading Environments with Moral Hazard. Chrysanthos Dellarocas (MIT)
[33] · Managing Brands in e-Business: an Experimental Study of Trust and Reputation Management. Claudia Keser (IBM Research)
14:45-15:00 Break
15:00-16:00 Wrapup Session: What the field needs now
Session Chairs: Chris Dellarocas (MIT) and Paul Resnick. (U. Michigan)

16:00 Symposium Ends

navigation
[ viewing | 50 entries back ]
[ go | later ]