I thought you might be using caller ID, so the number is only relevant to people with a specified phone number. (Similar to how Rebtel works.) But it doesn't seem to be the case here, as I think numbers can be shown publicly.
All phone numbers have some ambient activity -- wrong numbers alone account for at least some randomness, though some obviously have more than others, and even very established carriers sometimes distribute numbers with a "legacy". So it's sort of a known problem.
For Burner, each number goes through a quarantine and is monitored for activity before we use it.
EDIT: Also worth noting that SMS seems to be more popular than voice calls, at least initially. "Wrong number SMS" doesn't seem like an actual problem as of yet.
