Ryan at 37signals recently posted on the dangers of adding new features to software. Once a new feature is added, it is difficult to remove even if it doesn't work well and not utilized by the majority of your users. The problem is that SOME users will likely become frustrated when the feature is removed.
Ryan suggests that this should inspire you to be very careful about the features you implement. I think this advice depends a lot on the size/impact of the feature, but his point is valid.
The problem is - what should be done if the feature IS clunky and difficult to support? The application we recently deployed is designed to replace a couple of existing applications and processes that had been in place for years. It has a better underlying data structure, and a more streamlined user interface. But it also removes some features, like automatic faxing of reports to subscribers.
Will users be comfortable switching to the new system? Only time will tell. It was a risky decision and the customer spent time evaluating the alternatives, but ultimately for my customer, the decision was about limited resources and how to best serve their community.