Deferred Ayodhya visit to avoid legal hassles: Raj