Does only the tag need an IMU, rather than the anchors?

Answer

Yes. For the referenced configuration, the tag needs the IMU; the anchors do not need that feature.