Restaurant owners have expressed their frustration after road closures meant they lost out on business on Mother's Day.