Amazon Alexa's robotic voice is causing "deep distress" for dementia patients by telling them to take their medicine, a new report has warned.
The technology think tank, Doteveryone, said older social care patients were often left confused by the new gadgets and fearful they would replace contact with human carers.
It also found that disabled people feared that incoming adventures with smart homes that had features such as self-opening doors and windows could malfunction and leave them trapped.
The report, Better Care in the Age of Automation, argued that technology had a vital role in improving the care system but should not lead to "naive enthusiasm" that gadgets can replace human carers.
It is also called for government to invest in training up carers to use technological developments to take the strain off their workload so they could spend more time with patients.
The findings come as Hampshire County Council last year trailed giving social care patients Amazon Alexa smart speaker devices to help remind them to take their medicine and when their carers are coming.
Doteveryone conducted interviews with more than 100 care givers and patients across the country to see how they thought about technology to help the UK's struggling social care system. Alexa devices left dementia patients "often deeply distressed by an unfamiliar robotic voice reminding them to take medication".
Lydia Nicholas, the lead researcher in the report, said: "What was flagged up by care professionals and specialist nurses was if you were a dementia ward there is no point in having robots telling people what they would just get distressed and confused.
“Essentially, reminding someone to take their medication when you are in a social care context is not just a fact of flagging up three o'clock and just taking the time to take your drugs. A lot of the time it is about convincing someone who is quite distressed about the fact that their medication is safe or reminding them to get a glass of water, making sure they have eaten with it. ”
A similar concern raised in the report was that some gadgets left dementia patients confused. One interviewee said her elderly mother who had a condition thought her distress alarm was a button for her to let people know when she wanted coffee.
Another area explored was the growing phenomena of smart homes that let people various control functions in their homes remotely.
Ms. Nicholas added: “I spoke to people who said their smart home was fantastic, but they found the idea that they would be completely relied on terrifying. They were distressed at the idea that they could be trapped in a house that was getting hotter and hotter if they couldn't open the windows. ”
The think tank argued that the government and councils needed to be thoughtful about how tech should be used in the care system and that it was a “tool” rather than a replacement for human workers.
The report recommended that a new royal college for carers be create to help professionalize carer workforce and train people on how to use technology effectively in care.
It is also called for Enablement Panels to be created, to be run by disable people, for those in care and their families, to inform how new technology can be used in an effective way.
Rachel Coldicutt, CEO of Doteveryone, added: “Technology has the potential to be part of the solution to our current problems – but only if used responsibly and strategically. Doteveryone's latest report is a challenge to the naive enthusiasm for technology, applied as ad hoc digital sticking plasters, and is a call to create a robust care innovation strategy. ”