Predictim, an app that claims to vet babysitters by analysing their social media presence, is being investigated by Facebook and has been blocked by Twitter over concerns that it violates personal data policies.

The service, based in California, offers to scour a prospective babysitters social media using APIs and algorithms to provide a score out of five, suggesting how safe they may or may not be. According to its creators, the service looks for posts about drugs, violence and other “undesirable content”, and is simply automating the kind of checks that any member of the public could do.

However, earlier this month, Facebook revoked most of Predictims access to user data pending an investigation, concerned that the firm may be in violation of its policies regarding the use of personal data. Following its access being limited, Predictim claimed that it was still able to scrape public data to power its algorithms, a practise that is also against Facebooks terms of service. The social network is now considering blocking the firm entirely, a move that has already been made by Twitter.

“We strictly prohibit the use of Twitter data and APIs for surveillance purposes, including performing background checks,” said a Twitter spokesperson. “When we became aware of Predictims services, we conducted an investigation and revoked their access to Twitters public APIs.”

Predictim gained considerable exposure over the weekend when it was featured in a front-page story by The Washington Post. In the story, experts warned that relying on algorithms for background checks could result in messages being misinterpreted as harmful, or truly abusive content being missed.

“Kids have inside jokes,” said Jamie Williams from the Electronic Frontier Foundation, speaking to the Washington Post. “Theyre notoriously sarcastic. Something that could sound like a bad attitude to the algorithm could sound to someone else liek a political statement or valid criticism.”

Predictim said that it has a human review process to its system ensuring that every post that is flagged is checked to prevent false negatives. According to the service, it is able to spot instances “when an individual demonstrates a lack of respoect, esteem, or courteous behaviour”.