Here’s what the majority in the Sherrill case said in 1977 (citations omitted):
[W]e are presented with a situation where the White House has voluntarily decided to establish press facilities for correspondents who need to report therefrom. These press facilities are perceived as being open to all bona fide Washington-based journalists, whereas most of the White House itself, and press facilities in particular, have not been made available to the general public. White House press facilities having been made publicly available as a source of information for newsmen, the protection afforded newsgathering under the first amendment guarantee of freedom of the press … requires that this access not be denied arbitrarily or for less than compelling reasons…. Not only newsmen and the publications for which they write, but also the public at large have an interest protected by the first amendment in assuring that restrictions on newsgathering be no more arduous than necessary, and that individual newsmen not be arbitrarily excluded from sources of information…
In our view, the procedural requirements of notice of the factual bases for denial, an opportunity for the applicant to respond to these, and a final written statement of the reasons for denial are compelled by the foregoing determination that the interest of a bona fide Washington correspondent in obtaining a White House press pass is protected by the first amendment. This first amendment interest undoubtedly qualifies as liberty which may not be denied without due process of law under the fifth amendment.
In other words, (1) no, the president doesn’t need to call on a particular reporter but he can’t kick him out after inviting him in absent a good reason, and (2) even if he does have a good reason he needs to formally notify him first and give him a chance to respond. That’s the case law that Kelly was stuck with. And so:
Federal judge rules: White House must reinstate Jim Acosta’s press pass — for now