If the pipe is accessible (under a sink or in a crawlspace), feel for "ice-cold" spots or visible frost.
Dealing with a frozen drain pipe can be quite a challenge. Here are some methods to help you unfreeze it: how to unfreeze a frozen drain pipe