Text this: Wait a Second! How Delays Disrupt Optimal Policies