Angie's List is an invaluable resource for us in selecting contractors. Before they came along we were much less satisfied with contractors we'd hired, largely on recommendations from family, friends and coworkers who typically had based their judgments on only one job.
But AL is not a silver bullet. We learned that last summer when we contracted for a high-dollar-value home repair with a contractor who was a Super Service Award recipient for that year. After giving them thousands in down payment we struggled for months to get them to do the work they had promised. Along the way we found out they had outright lied to us about their qualifications, they "forgot" to do several things they had promised, and when we finally got the work done, it wasn't complete nor was the quality to our satisfaction. We exited the process a half a year after they had promised to finish, exhausted, disillusioned and grateful we hadn't lost our shirts, as it had appeared we would numerous times.
Throughout this harrowing experience we marveled that this contractor, who had only the most rudimentary business skills and a shocking inability to interact effectively with customers (or his own workers), even had decent Angie's List grades, much less a Super Service Award. But as the weeks wore on and we learned more about this guy we started to understand why his ratings weren't worse. We asked for our money back after he was three months late on his start date. He told us we'd get a refund only if we signed an agreement not to file an AL report. We were willing to do this, but he wasn't able to come up with our full deposit in the four weeks he'd allowed, so the cancellation fell through.
When the contract finally was finished, we filed an AL report that was fair (maybe "charitable" is a better description, as we didn't want to post anything that we couldn't document, even though a lot of things happened in phone conversations, etc, that were fairly blood-curdling) and gave them a "D" rating. Within a few days he sent word he was planning to sue us and he challenged our report with Angie's List, succeeding in having some crucial verbiage removed from our report. Angie's List did not even call us to try to verify his claim. AL based their redaction on some documentation the contractor provided that didn't prove the intended point. Angie's List didn't even examine the document—the redaction went in immediately and only then did AL contact us to inform us of the change. Within a day we rebuffed this effort with our own carefully kept documentation. Angie's List took many days to consider this, agreed the contractor's documentation was non-sequitur, and then took longer to retract the correction to our report.
Next we received a letter from the contractor's attorney threatening to sue us over our report. We reported this to the AL customer service staff, which offered to remove our report if we wanted, but provided no other relief. We held to our guns, knowing our report was fully documented. So far nothing has come of it, and our counsel has said that at least we would be entitled to recover our cost of defending such a suit. But AL has offered no assurances at all as to whether or how they would back us up.
Finally, the contractor posted a response to our report that attempted to undermine our report by a combination of confused reasoning, mis-stated information (like dates), irrelevant near-truths and outright false claims. The response, we felt, largely reinforced our original report. Several facts could easily be verified by a reader (such as licensing dates), some statements were clearly inconsistent and the response had an overall adversarial tone. But there were two statements that we felt represented a legal exposure for us, so we provided AL with a rebuttal and the relevant proof from our records. Again, it took AL weeks to verify our claims where it had taken them only a day or two to respond to the contractor. AL stated the delay was to give the contractor a chance to rebutt our evidence. Yet we were never given a similar chance to rebutt the contractor's "proof" earlier.
We felt that this, together with Angie's List's unsupportive attitude regarding the contractor's documented lawsuit threats and attempts to induce us not to file a report, showed a bias by AL in favor of the contractor in disputed reports. We also suspect a pattern by the contractor of intimidating or inducing dissatisfied clients not to file negative AL reports. This pattern was evidenced by statements the contractor made directly to us.in addition, a careful examination of previous responses showed that the contractor responded vociferously and with adversarial language to even the slightest negative rating—in one case making an angry response to a customer who rated him with all As except for one "B" for punctuality!
A few days after our objection to their response had been cleared and their response corrected accordingly, Angie's List awarded the contractor a Super Service Award for the previous season.
During this process we learned a couple of facts that might not be intuitive to the AL user:
1) When a subscriber files an arbitration request (AL calls it a CRP—an unfortunate acronym that we hope doesn't reflect their attitude about the process!), if the contractor and customer come to a satisfactory resolution of the dispute, the dispute AND THE ORIGINAL REPORT are taken out of the AL database leaving no record whatsoever of a highly dissatisfied customer's feedback. We think a shrewd contractor will intentionally "hold back" something in order to force a CRP, knowing that they need only satisfy their own contract (or less!) in order to have a negative report on them deleted.
2) Although Angie's List claims companies cannot buy their way onto the list, AL aggressively markets advertising (listing enhancements like coupons, etc) to companies with A or B ratings. AL is decidedly NOT entirely member supported. These advertising fees can be well over $1000 according to some contractors we've spoken with. At least one contractor we spoke with had been left with the (perhaps faulty) impression that he needed to pay in order to maintain visibility on the list. That myth is probably perpetuated by Angie's List using the term "Member Contractor" to describe contractors who have paid for advertising. Furthermore, any company that wishes to use the Angie's List SSA logo for any purpose must license the logo each year s/he wins it. And there are other ways Angie's list gets income from contractors.
We believe that such a situation, necessary though it may be to support the AL business model, will lead to some conflict of interest, and is probably at least partly to blame for the apparent contractor bias noted above. Clearly, AL will want to keep A and B contractors in the A/B category and paying SSA logo users winning SSAs to avoid undercutting income and alienating income sources. We think that just the business benefits of being listed with AL should be enough reward for a highly-rated contractor and that no compensation should come from contractors for ANY contractor-page content. Two contractors we spoke with said that Angie's List has driven their revenue so hard that they are deciding whether to expand significantly or find other ways of throttling demand. This leads us to believe that paying for advertising shouldn't be necessary for contractors to benefit from the system. The question is whether Angie's List can thrive without this income stream. At least Angie's List is credible as a rating service, unlike some competitors such as Merchant Circle.
Having said all of that, I have to reiterate that despite its imperfection we find Angie's List to be a very useful and worthwhile service and have spent literally hours writing reviews on contractors we've hired—most of them highly positive.
Still, after our recent experience we now realize more than ever that Angie's List needs to be the "cereal" part of a balanced breakfast—supported by numerous other sources of information on a contractor before signing happens. Some other suggested sources are the state licensing department, state and local consumer protection departments and organizations, state court records (many of which are now available free-of-charge on the Web), credit rating agencies, the Better Business Bureau, local contractor associations, municipal building inspector's office, local building material supply companies where the contractor would be most likely to have accounts and even the contractor's bank.
Contractor-provided references should be considered, but not for positive information as the contractor is certain to provide only references for his/her most successful jobs. One contractor sent us to four references, each of which was a family member or former employee!
Finally, I should note that the way Angie's List's database is constructed may tend to inadvertently conceal information about contractors. It is difficult (or impossible?) to get a single list of all reports for a contractor in every category in which he/she has done work. (AL, please read this sentence carefully again before you rebut.) Consequently, as we followed up our recent bad experience by digging deeper into Angie's List reports on the contractor, we discovered (in other categories) numerous bad ratings and contractor responses that would have given us pause had we discovered them before we hired the contractor. These included a relatively recent unresolved dispute that involved quite a large value.
When you examine an AL listing, it's important to look into EVERY category the contractor does work in, even if it's not the category you intend to hire. What you're looking for is the contractor's business sense, not just their craftsmanship. Also, go back more than the two years AL uses to determine the contractor's rating. Two or three "bad" reports in two years may not seem like much, but a dozen or more in four or five years may be a different matter, especially where the contractor's response has been adversarial rather than solution-oriented. As we learned, it is possible for a contractor with very poor business skills (or perhaps one that's teetering financially) to eke out a Super Service Award, especially if the contractor is especially aggressive about fighting or eliminating poor reports.
To Angie's List, we have a number of suggestions, some of which may, we know, be incompatible with your business model and the need to continue a service that is basically working.
1) Sanction so-called "Member Contractors" and Service Award contractors who "go after" unsatisfied customers by inducing or intimidating them not to write reports or who use the CRP process to have reports excised. I'd suggest immediately revoking the Member Contractor's advertising privileges for the next 12 months with no refund to the contractor or with advertising fees pro-rated to period following the 12-month sanction.in our view, trying to side-step the process in this way automatically says a contractor isn't providing "Super Service." So the latest SSA should be revoked summarily and the bullying contractor should not be allowed to state that he/she has ever received the award for that year.
2) Change the CRP process to ensure that the original bad feedback about the contractor must necessarily remain, but supplement the bad report with an update on how the contractor successfully resovled the situation, should it be resolved. We think a contractor who truly cares about customer service will see this as an opportunity to demonstrate to potential customers how well she or he can work to resolve customer dissatisfaction. We think Angie's List subscribers are smart enough to consider a well-resolved complaint as a positive comment on the contractor, as long as a pattern hasn't developed. Also consider whether it would make sense to allow the subscriber to post an update on the contractor after complaint resolution has occurred. This would be a chance for a newly-satisfied customer to praise a contractor who worked positively to resolve the complaint.
3) Reconsider whether contractors should be allowed to contribute income to Angie's List under any circumstances (including SSA logo licensing). Perhaps Angie's List is now so popular (in some cities, at least) that this contractor inducement can now be eliminated wihout affecting the positive image of Angie's List among contractors. At the least, the terminology "Member Contractor" should be rethought—it should be made crystal clear to everyone that contractors truly don't buy their way onto the list.
4) Make the reporting process consistent to ensure objections by both contractors and AL subscibers are given at least equal attention and verification, and that updates are posted only after both sides have had a chance to comment. We think this will reduce the amount of time Angie's List help desk personnel have to spend mediating updates or dealing with unsatisfied subscribers or contractors.
5) Perhaps a minor complaint, but the CRP acronym should be eliminated. Perhaps the process could be renamed Complaint Arbitration Process (CAP), Resolution Mediation Process (RMP certainly isn't as bad as CRP) or even something corny like "Win/Win Mediation, " WWM.
0 comments