Foursquare and the problem of SuperUser fatigue

UPDATE: As if on queue, things are suddenly looking up. :)

Foursquare’s greatest strength and also it’s greatest weakness is that its database is built and maintained by its users. Users can instantly add venues that the database does not contain, and easily make changes, such as adding a new phone number or submitting duplicate venues to be merged.

The advantage of this system is that new venues are rapidly surfaced by the community, making Foursquare’s venue database one of the most accurate available. The disadvantage is that the system can easily be abused – both intentionally (by people creating fake/spammy/illegal venues) or unintentionally (by inexperienced users accidentally creating duplicates or entering incorrect data).

To help keep the database clean, Foursquare employs so called “SuperUsers” – dedicated and passionate members of the community who help clean up the venue database. When a regular user submits a fix or change – SuperUser’s are the people who approve it.

In dense urban areas, where the Foursquare community is flourishing, this system generally works pretty well, since there are plenty of enthusiastic SuperUsers to help out. This makes Foursquare’s location database generally far more accurate than its competitors, but unfortunately not always – there are situations when it falls short. In small towns and rural areas where there may be no local SuperUsers the system breaks down, since fixes submitted by users are not being processed. Equally, in some large cities, there are simply too few SuperUsers to cope with the volume of work being created, and many problems remain unfixed.

Take for example, Kuala Lumpur, capital city of Malaysia. It currently has just over 140,000 issues awaiting moderation – that’s 140,000 requests that must be manually approved by the local SuperUsers. It is a daunting challenge for the local SUs, and only the most dedicated are plugging on. The rest are giving up and consider it to be, quite simply, not worth the effort. The incentives for them to continue just aren’t there.

Kuala Lumpur is just one example, but there are many other places across the globe with similar problems, let alone the countless small towns and rural regions where SUs do not exist. Furthermore, with many SUs calling it quits the problem is rapidly getting worse. As users continue to submit fixes and updates, there are fewer and fewer SUs approving them and the backlog increases.

I make no secret of being a big fan of Foursquare. I have been a user since the beginning, and I am big believer in their mission to become the de-facto local search provider. I also believe that they are the company best positioned to dominate in the local search space, in the long term.

I personally find Foursquare most useful when travelling. I used it extensively last year in Phuket, Thailand to seek out the best restaurants and it proved invaluable – way more useful than its competitors. Unfortunately, as discussed above, it is not perfect everywhere, and I wanted to see if I could figure out where the issue lies.

What’s causing the problem?

  • Careless users are adding new venues without first checking if one already exists. This has created many duplicates that need merging.
  • This is exacerbated in areas where there are lots of users, but few (or even no) passionate SUs working through the change queues. Vacation spots are a good example of this – lots of Foursquare users visit the area but there are no local Super Users to manage the venue database.
  • Many of the problem areas have had their database built mostly by visitors rather than locals, resulting in many inaccuracies and sloppy data entry.
  • Not enough venues are being claimed by businesses, who could enter accurate data for their venues.
  • Spammers are creating many fake venues.
  • Foursquare’s duplicate robot is generating far too many false positives.
  • There are not enough SuperUsers covering remote geographical areas.

What can Foursquare do to mitigate the problem?

  • Prevent new users from creating venues, until they have agreed to the rules. The first time a new user creates a new venue on Foursquare, they should immediately be sent an email containing guidelines on venue naming conventions, do’s and don’t etc. The email should request that they agree to the rules of venue creation, before their created venue is approved in the system (currently they can check into it themselves but it is not yet permanent). A link will take them back to the Foursquare website, where they must sign in and click ‘I Agree’. They will be then given the ability to edit and confirm the data of their recently aded venue, and from then on will be free to create venues on Foursquare. If they do not agree, they will be forbidden from creating any more venues until they do. In this instance their venue, should be held in queue for a few days, before being approved by an SU. Similarly, the first time they submit an edit of a venue’s data, they should also be requested to agree to the guidelines, if they have not already done so. This system will both educate new users on how to add venue data correctly, and also prevent spammers from creating new accounts and immediately adding lots of new venues. (Perhaps the button should include a CAPTCHA to prevent bots abusing it.)
  • Introduce “Foursquare Ambassadors” who work with local businesses to help them claim their venues and update their information themselves. Local business owners are notoriously non-technical. They need help understanding how Foursquare can help them, and how to claim their venue(s). Foursquare Ambassadors could be Foursquare’s employees “on the ground” working to improve the database in their local area.  Foursquare SUs should be able to apply to become Ambassadors for their local area, and Foursquare should introduce a rewards program for people who help local businesses claim their venues. Local businesses really have a motivation to keep their venue data accurate, once they undertsand how it works.
  • Queues should prioritise changes submitted by real users, before the robot generated changes are offered. People generally submit fixes for selfish reasons – they personally find the problem annoying and want to see it repaired. If users see their fixes approved quickly they will be more inclined to submit more fixes. By offering SUs the user submitted fixes for approval first, they’ll be approved faster, and people will see the results of their fixes more quickly.
  • Provide regular users with incentives for submitting merges, fixes and changes. Keep visible scores of the total fixes that they have submitted and have had approved, similar to what the SUs have. Maybe offer a level-up badge for the number of approved fixes. Once they reach Level 10 can automatically be invited to become a SuperUser.
  • Create a monthly SuperUser day where everyone is encouraged to fix the database in their area. How about on the Fourth day of every month?
  • Widen local SU reach in areas where merge queues are small. In areas where the merge queues are small, Foursquare should automatically widen the geographical area that SUs can work on to incorporate surrounding areas that may not have a local SU. For example, if a city has a lot of SUs who have worked through their merge queues, widen their reach to incorporate the surrounding towns and rural areas. If the merge queue size increases, reduce the area covered.
  • Encourage people to become SuperUsers in Small Towns and Rural Areas
    If a user is a local in an area that lacks an SU, and is an active user, invite them to apply to become an SU by submitting fixes. Email them with this suggestion, explaing how they can do this, and why they should. Create a SU coverage map showing where Super Users are needed.
  • Promote the Idea of Helping Out to active Users
    Foursquare should explain to active users why they should help out, and how they can help out. They should send out an email to new users, who are actively using the service, explaining how submitting fixes helps the community. By honest about it, tell them: “this community is built by people like you, and helping fix it improves it for everyone”. Create a “Foursquare Etiquette” manifesto, explaining what good Foursquare user behaviour is. Send them Tips on how to help.
  • Foursquare needs to offer better incentives to the SUs in problematic areas. How about upgrading trusted SUs to paid content managers, whereby they receive financial compensation for their hard work? Or, recruit trusted SUs from other areas to help out process the larger queues? Or, offer Foursquare sponsored discounts/specials at local businesses for the top performing SUs?

How can you help?

As Foursquare transitions to become the location layer of the internet and the leader in local search, they need your help to improve their database. They currently have the most accurate location database available but it still has a long way to go. If you are a user of the service, why not help out occasionally? It improves things for everyone. Start with your local neighbourhood. After all, you know it better than Foursquare does – for now. ;)

If you are a Foursquare user interested in applying to become a Super User you can apply here. It is a good idea to familiarise yourself with the guidelines here first. If you have already applied, keep submitting venue data fixes and duplicate merges, and be active in the community – it’ll help your chances of getting approved. For more tips please see my previous post here - it contains everything you need to know on getting the most out of Foursquare.

  • Ryan Wallace

    Great work, to put together a list like this shows your loyalty and passion about improving the 4sq community. Here are my responses to your ideas (1-9). Note that my opinions are coming from a US bias, whereas yours are from an international bias.

    (1) I think you are on the right track, but your chosen method could be overly prohibitive. [BRAND NEW VENUES] Giving all users the ability to create venues without any authorization process is one of the primary reasons why the database flourishes. You said it yourself: “new venues are rapidly surfaced by the community”. If we require authorization steps for entry level users before creating venues, I think there is a high likelihood of many users simply not submitting new venues. I would personally rather have several users innocently create duplicates of the same new venue (that are easily caught by Jimmy F or the SUs) than restrict users and risk the venue not show up in the database as fast as possible. [WELL ESTABLISHED VENUES] When you attempt to add a new venue on the web (https://foursquare.com/add_venue) and provide values that are similar to venue that already exists then you get the warning message: “Hmm… there’s a chance we already have [Venue Name] in our database. Double check the listings below. If it’s missing from our list then {Add Venue}.” (I do not recall what SU level is required to access this, but I don’t believe any is needed). I think a similar workflow should be added to the mobile platform (where 99.9% of the venues are created). Theoretically this is already handled by the fact that you have to search (from the check-in page) before you can add a venue. Given that this does not weed out enough duplicate venues, there is definitely room to improve this process. [SPAMMY CREATORS] I think the 4sq engineers are more than capable of working some algorithm magic to weed out users who habitually create venues that are later marked for merger or flagged as inappropriate (and now probably an inordinate number of private venues). These users should be sent an email reminding them of the guidelines and their ability to create new venues should be restricted (such as your suggestion that they need to be reviewed by SU before going live), suspended for some time period, or removed completely. The same system should be implemented for abusive tipsters. [SUGGEST EDIT] I do like your idea to add links to the Style Guide (http://foursquare.zendesk.com/entries/214565-is-there-a-style-guide-for-adding-venues) and/or Editing Guidelines (https://foursquare.com/info/editguidelines) on the Suggest an Edit dialog. This should be done immediately. (Side note, those two documents should be combined into a single page to avoid duplicative and confusing info).

    (2) I agree this definitely needs to be part of 4sq’s natural evolution. I am sure there are many third-party marketing/advertising experts that are already providing this service to local businesses, but I think we all agree that most local businesses are missing the boat on promoting their business using 4sq. I have, several times, found my self discussing 4sq with a local business owners/managers, encouraging them to claim their venue. I am not sure that 4sq really needs to have paid employees. I think there are already plenty of enthusiastic SUs that are more than capable and qualified to help promote the 4sq brand. I envision this more as people like you or I are designated as ambassadors. We are given some additional training on claiming venues and some swag and window clings to hand out to businesses. We would assist in describing what foursquare is, why its important they claim their business, assist in claiming business, and explain/show how to make changes to business listings. I don’t believe that ambassadors should give advice about setting up specials, as most will be under qualified in providing that type of business advice, but we could easily refer them to specific individuals/teams at 4sq HQ that could help them with that.

    (3) I think this is a no brainer. While most of the Jimmy F flags are important, more credence should be given to suggestions submitting by actual people for the reasons you stated.

    (4) Getting their edits to the top of the pile and actually seeing the changes faster will be a really large incentive, but the badge idea is not bad, and I think showing stats on profile of how much they have helped out (edit count or similar) is also good idea.

    (5) I think you have hit on something REALLY important. Right now, there is a very sterile feel to the SU process. There needs to be more done to make the editing more social and even greater sense of community. I think a monthly SU day is a good idea. Another great one that I LOVE has been floated by Yinon over at 4sweep (http://www.4sweep.com/). Leaderboards/shared cleanup tasks, which would be likely weekly shared goals of the SU community “Clean up venues marked as Zoos” or “clean up venues that contain the word ‘work’” or “Everyone focus on Dallas, TX queues”. This sense of community and competition would benefit the database and the SUs

    (6) I would have to remember what is is like for SU1, but know for sure that SU2 can skip to any city level queue they want, they just have to enter the different city. In the US you can also go by zip code. In order to cover more ground (at least in rural US) SUs could simply enter additional queues. I would also like to hear 4sq admins thoughts on allowing all users to access country queues. An SU1 may be just as if not more enthusiastic that SU2/SU3, and although their votes to not carry as much weight and they do not have access to all the queues I can’t really think of a reason why lower level SUs should not be able to access country queues.

    (7) I like this idea, and wonder why we have to limit it to rural areas and small towns. Any user that has a sustained level of activity (say an average of one a day for a month) should get an email thanking them for being an active user, and encouraging them to use their local knowledge to help maintain the venue database in their area.

    (8) {comments from 7 apply here too} I think 4sq definitely needs to combine the Style Guide and Editing Guide into a single document. I would stray away from “manifesto” and stick to guide. Remember that 4sq has learned many things from that way that users (and SUs) have applied the guidelines locally, regionally, and nationally. These can’t really be viewed as set-in-stone rules and need some flexibility to vary.

    (9) I think 4sq can do plenty to show their appreciation to users without having to pay them. Creating a social/competitive aspect to SU duties will help with a lot to help with problematic areas. There is only so much that “outsiders” can do to clean up other areas. I can certainly help throughout the US and many other regions, but am likely less than useful in Japan, Thailand, etc. The only thing 4sq can do is promote more SUs within those countries to help combat the queues (which it appears they are on track to do.

    I have compiled a list of more technical items I think that 4sq needs to address, feel free to comment: https://docs.google.com/document/d/1_zNkRWLBGbbbnXG0USqtnVfvIyJjaQGgBC42vpSrBAg/edit?usp=sharing