Rhode was at the helm of the company for eight years.
During his tenure at the firm, one of North America's largest packaged food companies, ConAgra was transformed from 90 independent operating companies into three focused business segments serving customers in retail, foodservices and ingredients.
Meanwhile, the company divested commodity-based businesses, including fresh beef, pork and chicken, canned seafood, cheese and agricultural inputs to concentrate instead on marketing, operational efficiency and business consolidation with the aim of improving customer service and increasing margins.
The search committee will be led by ConAgra board member Steven Goldstone, retired chairman and chief executive of RJR Nabisco, according to board members.
In March, the company said its third-quarter operating profit fell 12.6 percent, to $396 million, compared with the year before, because of weak results in refrigerated branded meats, food production problems and inefficiencies in technology consolidation.
Analyst John McMillin of Prudential Equity Group in New York said on Wednesday that ConAgra needed a leader with operations expertise to replace Rohde, who is an attorney.
"He's a dealmaker," McMillin was quoted as saying by the Associated Press. "Now it's time to block and tackle."
He also wondered whether Rohde's announcement raised questions about the possibility of fundamental problems at ConAgra.
"Costs remain high and there could be earnings disappointments," said McMillin, who has questioned the quality of ConAgra's brands and criticized Rohde's performance.