Thursday, 25 June 2009

Mid-Summer Blog Post

By way of explanation...

I am very aware that I have not written something on the European Elections, as promised in my last post. In my mind I am having a big debate in recent weeks about where this blog is going & what I want from it. Should it concentrate on current, day-to-day stuff or be a bit more about political theory and big ideas? Should I make a serious effort to be more cultural in my posts? What is the blog trying to achieve and who is it meant to reach out to? (Apart from present company!)In short, I need a re-think and I need time to do so. Otherwise there would be no point in carrying on.

I am off to Nice in a couple of weeks time for my annual holidays. Going to visit Monaco (to see if it does seem smaller than Hyde Park!), Cannes and maybe cross over into Italy. One big consequence is that I am currently re-learning my French (1981-86 GCE O Level Grade D; 1998-99 GCSE Grade A) so that I can survive without constantly saying 'Parlez-vous Anglais?'. As I want to make a decent fist of this, it is taking up a lot of my spare time. Consequently, I'm leaving the big and final decisions about my blog until after my grand vacance.

Nice at Night- Bigger Than Hyde Park..

Refreshed and relaxed, I hope to start again seriously with this blogging lark towards the end of July. Before I go, I do hope to post my short story for the latest (well, the first since early 2007) short story competition at work. I will wait until the result (or at least until voting has finished) before it appears here. I think it is good- but I would say that!

In the meantime (and one of the reasons that my guilty conscience forced my to put something up here today), I saw this today about the nature of blogging. It made me think: just by keeping going, I am a blogging success!

The long tail of blogging is dying: The popularity of blogging seems to be fading as people turn to the easier aspects of social media: status updates and tweeting
Charles Arthur, The Guardian, 25 June 2009

Blogging is dying. Actually, no, let me qualify that. The long tail of blogging is dying. I say this with confidence. That confidence is based on two things: my anecdotal, but wide-ranging, analysis of what and how people remark on content from this section, and the surveys carried out by Technorati – which provides the Guardian with the feedback data that appears on our web pages. The interesting question is, what has replaced that blogging?

First, to my anecdotal analysis. Since the relaunch of this section in September 2005, we've included blog pingbacks among the "letters" section, recognising that people might not want to write a letter, or even send us an email (to when they have an opinion about something we've written. Instead, they might write a post on their own blog. Since January 2006 I've automated the process of searching for those blogposts. (Put simply: collect the links from the section – which I do anyway, stored on a MySQL database on my laptop – then input them into blog search engines via a preconfigured script.)

I'd then follow these posts. There were always a certain number of splogs – spam blogs – which simply pick up keywords and dress them with Google AdSense adverts. Boring, but easily ignored. Then there would be the blogs that copied entire articles – annoying (it denies us page views and advertising revenue), though usually they're easily persuaded not to persist. Then there would be those who had read the piece and had comments and insight, which we could extract for inclusion in the Letters and blogs section. Of course, the number of physical letters we received about what appeared here was absolutely tiny: months would go by without any arriving. Email remained the principal source, though blogs quickly began to make up the majority of content.

But recently – over the past six months – I've noticed a new trend: fewer blogs with links, and fewer with any contextual comment. (I'm defining a blog here as an individual site, whether on Blogger or Wordpress or an individual domain, with regular entries.) Some weeks, apart from the splogs, there would be hardly anything. I didn't think we'd suddenly become dull. Nor was it for want of searching: mining for blog comments, I use and Google's Blogsearch.

Where is everybody? Anecdotally and experimentally, they've all gone to Facebook, and especially Twitter. At least with Twitter, one can search for comments via – though it's still quite rare for people to make a comment on a piece in a tweet; more usually it's a "retweet", echoing the headline. The New York Times also noticed this trend, with a piece on 9 June about "Blogs Falling In An Empty Forest", which pointed to Technorati's 2008 survey of the state of the blogosphere, which found that only 7.4m out of the 133m blogs it tracks had been updated in the past 120 days. As the New York Times put it, "that translates to 95% of blogs being essentially abandoned".

I see it: NetNewsWire, my RSS feed reader, has nearly 500 feeds. When one of them hasn't been updated for 60 days, it turns brown, like a plant dying for lack of water. More and more of the feeds I follow are turning brown. Why? Because blogging isn't easy. More precisely, other things are easier – and it's to easier things that people are turning. Facebook's success is built on the ease of doing everything in one place. (Search tools can't index it to see who's talking about what, which may be a benefit or a failing.) Twitter offers instant content and reaction. Writing a blog post is a lot harder than posting a status update, putting a funny link on someone's Wall, or tweeting. People are still reading blogs, and other content. But for the creation of amateur content, their heyday for the wider population has, I think, already passed. The short head of blogging thrives. Its long tail, though, has lapsed into desuetude.

PS Another conundrum I am wrestling with is how to integrate my Facebook postings and musings with those of this blog...Tootle pip for the mo!

Wednesday, 10 June 2009

Bits on books, publishing, bookshops & writing

I will turn to the European/Local election results and the implications for British politics/the next General Elections in the next few days, as I am still trying to think matters through.

In the meantime I'm bringing together a few articles about writing and the world of books. Without sounding too alarmist, it seems that the traditional business models for writing, publishing and selling books are going through a challenging period. This surely cannot be blamed on the existence of the Internet and economic downturns alone. Moreover, I'm not too enamoured by the thought of e-books. I like the fact that with dead tree (hopefully recycled) books you can fold pages, scribble in ink, pencil or marker pens or read them in the bath or on the toilet without much chance of getting electrocuted. However, perhaps I am a member of one of the final generations to think like this.

Anyhow, onto the articles. First of all, have we seen the death of the professional writer, or at least of the writer who can live by writing alone? Ian Jack seems to think so.

The age of the gifted amateur has returned: The woes of publishing make it easy to forget that Fielding, TS Eliot and others were part-timers
Ian Jack, The Guardian, Saturday 2 May 2009

We are in the twilight years of a certain kind of paid employment: the business of inking words on paper, to be read by a large audience that is largely unknown to the author. The crisis in newspapers is especially acute. But neither is book publishing immune. Advances against royalties are tumbling, staff have been cut, publishers take far fewer risks. The recession offers only a small part of the explanation. The fact is that generations are now growing up with the idea that words should be read electronically for free - a new human right - which has grave consequences for the people paid to compose and edit them. Writers and journalists like me, old enough to know manual typewriters, tend when we meet to congratulate ourselves on having seen "the best of it", meaning the years when a career could be based solely - mortgage secured, lunches enjoyed, etc - on small or large acts of English composition, often flawed.

This way of living reached its full bloom only recently, between the late 1980s and early 2000s, when the enormous expansion in newspaper pagination and a burst of new lifestyle magazines increased the demand for wordage and publishers bid against each other to pay out large advances (the sums look incredible now - £500,000 for a first "literary" novel) for no other reason, it sometimes seemed, than to give an editorial director crowing rights over his rivals. What nobody considered (certainly not me) was that paid authorship rested on certain technical, social and legal developments, old but far from ancient, that were about to be undermined and overtaken by a new technology that was much more democratic. In other words, that paying an author to read his work wasn't an unchallengeable habit set in stone, like buying bread from a baker, but the result of two German inventions, moveable type and the rotary press, and two British ones, copyright law and that first large audience for print known as "the reading public".

Before Gutenberg devised his letterpress, literary production lay in the hands of scriveners, and writing was a spare-time hobby sustained by the patronage of the church or aristocrats. For a long time, the printed book did little to change this. Among writers, the play offered the best chance of a full-time writing career because a play had paying audiences - Shakespeare got a tenner for Hamlet whereas more than 60 years later Milton earned only a fiver from the first printing of Paradise Lost. By the 18th century, reading had spread from the aristocracy to the bourgeoisie. Alexander Pope received £5,000 for his translation of the Iliad when an agriculture worker's wage was 10 shillings (50p) a week, while Daniel Defoe wrote his way out of debt and had enough left over to set himself up in a country house in Stoke Newington.

But these were exceptions. Patronage declined, and the writer at the end of the 18th century was often poorer than his equivalent had been at the beginning. Books were luxury items. A literate craftsman on £1 a week was hardly likely to fork out 13s 6d for a copy of Tom Jones, which meant that its author, Henry Fielding, depended for his living on his post at Bow Street magistrates court.

Only with the swelling of the British middle class did writing become a real possibility as a career. In 1812, the editors at the Edinburgh Review reckoned that there were "probably not less than 200,000 persons who read for amusement and instruction, among the middling classes of society". The Review paid its contributors well; it was anxious to establish the writer as a professional. Walter Scott is the most famous example of a new breed, exchanging a legal salary of £1,300 for a series of furiously written novels that sometimes earned him £20,000 a year, a lot of which went into building his Gothic castle in the Scottish Borders. By the time Dickens, Balzac and Dumas were producing their highly profitable serial novels, the triangular foundations of the modern book trade - author, publisher, bookseller - had already been laid. Writers got advances against unwritten work and then flattered booksellers at "trade dinners" to persuade them to up their orders. A bestselling author could make a small fortune for his publisher: Byron for John Murray, Dickens for Chapman & Hall, the word of God for William Collins, who bought a country house and a steam yacht by selling 300,000 bibles a year.

Until now, remarkably little about literary production has changed. Some authors - JK Rowling is the modern Scott - generate big profits, while others - let's think of Ian McEwan as Byron - certainly earn enough to keep wolves from their various doors. A misleading idea has arisen, however, that writers generally can earn enough money to do nothing else. The idea is ignorant of history, of TS Eliot keeping himself comfortable on academic stipends and a publishing house directorship, of Angus Wilson superintending the reading room at the British Museum. It may be that we have it because authorship is now so visible, with the author turned into a small celebrity. But we can all be authors now and publish ourselves on the web. What you might call the moral and aesthetic case for writing - to think, imagine and describe and then communicate the result to an audience - can be satisfied online. It just doesn't make any money. The age of the gifted amateur is surely about to return.

At British and American universities, this future has to be kept as a woeful secret. A great paradox of the age is that while newspapers continue their inexorable decline and publishing cuts its costs, journalism and creative writing degrees have never been more popular. Year on year, journalist applicants stood a quarter higher at 13,229 for courses beginning this autumn. Creative writing can now be learned at nearly every British institute of higher learning. Figures are hard to come by, but Britain is probably turning out about 1,300 "creative writers" every year.

Why do young people apply? Because they think they can be the next Zadie Smith. Why do universities encourage them? Because money can be made from fees. Is this responsible behaviour? We need to weigh the smashed hopes of creative writers against the financial needs of their tutors, who are themselves writers, and earning the kind of money that writing would never supply. A closed little dance: tutors teach students who in turn teach other students, like silversmiths in a medieval guild where a bangle is rarely bought though many are crafted, and everyone lives in a previous world.

So if the world of the professional full-time writer is under threat, whither the book and book industry?

The decline and fall of books: Traditional bookshops are closing; vending machines are churning out novels; and e-books are the new paperbacks; so is this the final chapter for the book industry?
Nicholas Clee,The Times, May 7, 2009

Like so many prototypes of supposedly revolutionary inventions, the Espresso Book Machine (EBM) fails to impress. Sited in a branch of Blackwell's in Charing Cross Road, London, the machine resembles an oversized photocopier with extra bits. Can this be part of, as a Blackwell's executive has claimed, “the biggest change since Gutenberg”?

It is an attractive idea. Wouldn't it be marvellous to go into a shop knowing that if the book you wanted was not in stock, you could get it printed specially for you? Or that you could browse the catalogue and get copies of whatever you fancy in minutes. Blackwell's claims that the EBM offers 400,000 titles, which are digitised texts from libraries and other sources. On receiving an order, the machine takes about 20 minutes to set up the file and then prints a perfectly acceptable paperback book in five minutes. A 400-page book costs about £9. Those produced from digital files look good; those produced from books that have gone through a scanner look a bit rough.

On May 6, Amazon unveiled a new Kindle e-reader (a widescreen version of the handheld device for reading electronic books). “You never have to pan, you never have to zoom, you never have to scroll. You just read,” said Jeff Besos, Amazon's chief executive, at the New York launch. It is yet another indication that the book industry could do with a new way of distributing and selling books.

Booksellers are struggling to make a go of shops with large ranges of slow-turning stock, and they are paying prohibitive rents. Leading publishers are gambling scary sums of money on books by top authors and celebrities and struggling to underpin their lists with less glamorous titles. Authors, unless they are talented or lucky enough to make the bestseller lists, look on as these firms' ever fiercer battles over dwindling margins eat into royalty payments.

No doubt future generations of the EBM will be less clunky. Then the machine will be well adapted to conform to the theory advanced by Chris Anderson, the author of The Long Tail. Businesses would thrive, Anderson argued, by supplying deep ranges of items, many of them selling in small quantities to niche audiences. Amazon is an example. The internet retailer promotes bestsellers heavily, with deep discounts. But it really scores in offering a huge catalogue of titles - many more than a terrestrial bookshop can stock. Even more profitable is Amazon's Marketplace business, through which it acts as a middleman for sellers of second-hand and out-of-print books.

The effects of Amazon's activities can be seen all around Blackwell's on Charing Cross Road, once the heart of London bookselling. Now only Foyles, Borders and Blackwell's remain. Blackwell's specialist neighbour, Sportspages, is long gone. Another specialist, the crime and science-fiction bookshop Murder One, closed recently, blaming competition from the internet. The art booksellers Zwemmers and Shipley have disappeared too, and the women's bookshop Silver Moon survives as a department within Foyles. Murder One was a great shop to browse in. But even the most conscientious supporter of independent bookselling must have found Amazon too tempting to ignore. Buy from Amazon and you start getting recommendations of new titles. If you made a trip to Murder One you could not be confident that, for example, every title in Andrew Taylor's Lydmouth series would be there; they are all on Amazon, though some are second-hand.

A visit to Blackwell's on a weekday afternoon illustrates another problem for terrestrial booksellers. There are fewer than ten customers in the shop. They have the run of thousands of square feet of selling space and of thousands of titles. Many of these titles attract buyers just two or three times a year; and those long-awaited sales may bring in no more than the price of a paperback. Meanwhile, Blackwell's is paying premium rents.

Booksellers have managed to make profits from this bizarre business model by balancing new titles, which sell quickly, with deep ranges of stock. The bestsellers pull in customers and generate instant turnover and profits; the backlist ensures continuing custom. The problem now is that profits from the bestsellers have sharply declined. The abandonment of retail price maintenance on books in 1995 allowed supermarkets and Amazon - the only retailers to achieve substantial growth in their book businesses in the past few years - to discount bestselling titles at levels that specialist booksellers struggle to match. As a result, independent bookshops have largely given up selling celebrity memoirs and other mass-market titles. Waterstone's, as the biggest bookseller in the country, has to try to compete, but by offering far deeper discounts than it would like. In its last set of financial results, Waterstone's profit margin was less than 3 per cent. Its latest strategy is the “Discover” campaign, an effort to promote its shops as unrivalled places to find expected or unexpected gems.

From the producer's end, the economics of the book business are just as challenging. The leading publishing houses are international giants, hungry for bestsellers, which are very expensive to acquire. Last autumn, Transworld had a huge hit with Paul O'Grady's memoir At My Mother's Knee. This spring, the company signed up a further volume from the chat-show host, but reportedly had to double its previous advance, to £2 million - because O'Grady, or his agent, wanted to match the sum that Dawn French had received for her memoir, Dear Fatty. So Transworld has paid twice as much for a book that will probably sell half as well. Another company found itself similarly penalised for success: to buy a second novel by the author of an unexpected bestseller, the publisher was asked to pay a sum equivalent to the profit it had made on her first book. It meant that the new book was almost certain to be unprofitable unless it reproduced the earlier success. It didn't.

In spite of the scary economics, celebrity books and other blockbusters have made a lot of money for publishers in the past few years. Will our appetite for such books be as strong in a credit crunch? It must be doubtful. Publishers are praying that the trend can survive, however, because the economics of the rest of their businesses is also looking dodgy.

The publishing model has a lot in common with the bookselling one: in theory, the bestselling titles bring in the turnover and profits and the rest of the list provides the bedrock of the business. But publishing the so-called “midlist”, or any book that does not come with some kind of marketing angle, has become more difficult. A book has to be “promotable”: the author will be young and attractive, or have an interesting CV; the work will tell an extraordinary story. Biographies of literary figures outside the top rank, or novels with undemonstrative virtues, do not cut it.

The reason why Tindal Street Press - an Arts Council-funded imprint in Birmingham - was able to publish Clare Morrall, the Booker Prize-shortlisted novelist, and Catherine O'Flynn, the Costa award-winning writer, is that no big publisher saw the potential of these authors' novels. (They do now: Morrall is with Sceptre and O'Flynn with Penguin.) Some acclaimed authors have moved to small houses: Maggie Gee (formerly HarperCollins), for example, is now with Telegram Books, and Tibor Fischer (formerly Chatto & Windus) is with Alma Books.

Practices that have been normal in the book industry for years are becoming unsustainable. You pay an author an advance - say £15,000 - that is probably too large, even though it is too small as recompense for what may have been a year's work. You print 1,000 hardbacks and manage to sell 800 copies to booksellers. Lorries bring them to your warehouse and take them out again to the shops. The book gets a few reviews, but no recognition from prize jurors. Half of the copies come back and they, along with the 200 that never left, get pulped. A year later, the paperback comes out. Richard and Judy fail to recommend it, the booksellers do not select it for their three-for-two promotions. The lorries go to and fro again and more books get pulped. No wonder publishers are looking for new authors through low-cost ventures such as HarperCollins' Authonomy, a self-publishing website, and Macmillan New Writing, which pays no advances.

This is where digital technology, such as the EBM and electronic devices, including the Sony Reader, comes in. Printing thousands of books that sit in warehouses or on booksellers' shelves, only to be pulped, is unsustainable. But remember the long tail: there may be a demand, albeit “niche”, for these texts. It makes sense to create digital files that can be downloaded or printed according to demand.

This prospect causes book industry figures to look with alarm towards their counterparts in music, where digital downloading, legal and illegal, has caused turmoil. It seems that consumers expect digital books to be cheap. But giants such as Penguin and HarperCollins will not be able to remain profitable if all their books cost less than, say, £10; and authors' rewards from these books, unless sold in large quantities, will be trifling. Worse, many consumers expect digital content to be free. While digital piracy is a minor problem for the book industry, it will grow.

Perhaps the digital revolution will decentralise publishing and bookselling, as Gutenberg's movable type did in the 15th century. Publishing, printing and bookselling may come together again. Will the bookshop of the future consist of a few hundred bestsellers and a print-on-demand machine? At Blackwell's, such a prospect required an imaginative leap. The EBM would print only some out-of-copyright works and those only if purchasers knew exactly what they wanted. The customers' terminal in the store was not functioning: you had to ask the bookseller to search for a title. The copyrighted works that various publishers are making available had not yet come through, and there was no search function on Blackwell's website.

A Gutenberg-style revolution is not, on this evidence, expected in the next few months. But if you are a lover of well-stocked bookshops, then you should enjoy them while you can.

Nicholas Clee is the joint editor of the book industry newsletter BookBrunch and the author of Eclipse (Bantam Press)

Perhaps it is not the future of the book we should be worried about, but the future of the serious book:

You can't be serious :Celebrity memoirs, declining libraries, the web and now the recession have all spelled bad news for publishing. Once thriving genres such as political and literary biography are ailing. Is it the end for quality non-fiction? Andy Beckett, The Guardian, Saturday 16 May 2009

Imagine dropping that in the bath...

Colin Robinson has been in publishing since 1976. He has worked for fusty companies and radical ones, for earnest independents and empire-building corporations, for Britons and Americans: as an editor, always involved in the slightly precarious business of putting out serious books. But recently he started noticing something about the way books are treated that disturbed him. "Here in New York" - Robinson lives in a fairly intellectual part of Manhattan - "books are quite often left out in the street. If people are moving, they don't take their books with them."

There may be a harmless explanation. Manhattan apartments are small. Some people always get rid of books once they've read them. Yet Robinson has some cause to see the phenomenon as a symptom of something ominous. On 3 December last year, despite what he describes as an editorial list "filled with erudite, well-written books", he abruptly lost his job at the American publisher Scribner.

So many other editors were sacked in New York that day, it almost instantly became known in the closely connected worlds of American and British publishing as "Black Wednesday". In recent months, such culls have become grimly routine in many industries. But among those who write, publish and sell serious non-fiction - the biographies, histories, travel and science books researched and written with a degree of subtlety for a general audience - the bad news seems to have been building up since long before the current recession.

The range of titles stocked by British libraries has been falling for decades. The net book agreement, which in effect subsidised the British book business, has been dead for a decade and a half. In that time, book retailers have concentrated increasingly on the genres that are easiest to sell. Book prices have collapsed. Within many publishers, sales and marketing considerations have come to trump editorial ones, and most authors of serious non-fiction have had to accept smaller advances and smaller print runs.

Meanwhile, review space for their books in most newspapers has shrunk. The time their work spends on the shelves of bookshops has shortened. The competition for readers' attention - from the internet, from secondhand books sold online, from the seemingly ever-expanding celebrity culture - has sharpened. Bestsellers and "brand-name authors" squeeze out less established titles and writers more than ever before. Supermarkets and chain bookshops squeeze out independent booksellers. The number of books published in Britain - well over 100,000 a year and growing, five times as many as in 1970, which is far more than in comparable countries - means that all books fight for air. The computerisation of British bookselling, more advanced than almost anywhere in the world, casts an increasingly cold eye on serious books' commercial performance.

At the same time, wider cultural shifts - the evaporation of the idea of a literary canon, a less deferential attitude to experts, changing reading habits in the digital age - may be making serious, would-be definitive books less attractive to a broad public. Once-thriving serious genres such as political memoirs, literary biography and literary travel writing all appear to be ailing.

"The fat years of the printed word are over," says John Sutherland, the academic and author of several books on the history of publishing. "Even if books get dirt cheap, readers simply don't have the time or motive to invest in them. The old cultivated readership is not as solid as it was. The safe library sale doesn't exist any more. There's been a loss of authority in the serious book." A former bookseller who is now a freelance literary publicist says: "There are plenty of good books going missing. Books that take five years to write. Publishers used to put them at the front of their catalogues. Nowadays the print runs are tiny for these books, about 2,000. Publishers say they can print more copies, but if they're printing 2,000 of something they're not going to get behind it. Because of publishers' falling profit margins, production values have gone down on some of these books. You're seeing paper that's turning yellow before it gets out of the shop. You've got publishers and literary agents blaming the bookshops and vice versa. You've got people going to literary festivals who'll pay £10 for a ticket to an author event but won't pay £20 for a history book."

Neil Belton, an editor at Faber with a long track record in serious non-fiction, is almost as bleak: "The book trade and publishing industry has embraced its inner philistine. The bigger book chains have semi-withdrawn from interest in serious books. The number of publishers that are committed to trying to bring these books to an audience is smaller. When they are interested in serious authors, the big publishing conglomerates are often chasing only the very big names, people established in their fields." The literary agent Peter Straus, previously a publisher at Picador, is also worried: "It is more and more difficult to place good books. Retail's changed. Advances have come down in the last two years. So many books haven't sold. There are too many books published. The harsh realities of the market will impinge on certain writers, certain publishers, certain agents."

In his 2000 polemical history The Business of Books: How International Conglomerates Took Over Publishing and Changed the Way We Read, the veteran American publisher André Schiffrin calls this process "market censorship". But one person's market censorship is another person's market realism. Clare Alexander, like Straus a well-known agent previously involved in serious publishing, gives an example. "Between about 2003 and 2006, a lot of agents and publishers thought history was the new rock'n'roll. The advances people were paid were ridiculous: £100,000-plus for books that were not going to sell. Now the same sort of books are getting probably about £30,000. Some of the advances in serious non-fiction used to be seriously out of kilter."

Until 2006, Scott Pack was the head buyer at Waterstone's, Britain's most dominant bookshop chain. "Historically, publishers have published too much of this [serious] stuff," he says. "Fifteen years ago, there were tens of millions of pounds of unsold stock sitting in bookshops. A publisher would put out a book on Henry VIII, say, and distribute 10,000 up and down the land. Then the publisher would be sitting there saying, 'We sold 10,000 - didn't we do great?', when really only 2,000 copies of the book ever sold. Nowadays bookselling is more professional, more commercial. It's not as nice ... A bookseller can return unsold books to the publisher after three months. Or if the book is in a three-for-two promotion, after four weeks ... But we see much better what the true sales of serious books are."

Pack argues that those who fear for such titles in the modern marketplace are being snobbish and pessimistic. "In the last 10 years, the British book industry has been selling more books. More people are reading than ever before. Some of those extra readers are buying the kind of books you find in supermarkets - misery memoirs, mass-market crime - so, yes, the market share for heavyweight non-fiction will be smaller ... There used to be a lot of noise around these books. They were books made for great reviews. But people didn't want to buy them."

Currently writing a book blog called Me And My Big Mouth, Pack is practised at making populist, intellectual-baiting arguments. Elsewhere in our interview, he dismisses upmarket publishers and reviewers as "the intelligentsia" and unsold books as "dead stock" and "rubbish". Yet he insists that it is still perfectly possible for a good, serious book to find a readership: "The vast majority are stocked in ones and twos. They can stick around, pick up word-of-mouth, sell steady."

How such gradual success can be achieved given the ruthless stock control of the book chains is not something Pack explains. Yet some publishers of serious titles share his optimism. "The market for really good books has not diminished," says Stuart Proffitt, the publishing director of Penguin Press, identified by Schiffrin and others as a rare example of a corporate-owned publisher making a success of upmarket books. Proffitt cites Rosemary Hill's erudite biography of the Victorian architect Augustus Pugin, published by Penguin to great acclaim in 2007. So far it has sold more than 17,000 copies in hardback and paperback combined, according to Nielsen BookScan, the US market research corporation that records British book sales. David Kynaston's history of the Attlee era, Austerity Britain, has sold 65,000 copies; Kate Summerscale's reconstruction of a Victorian murder investigation, The Suspicions of Mr Whicher, more than 305,000; Richard Dawkins's anti-religious polemic The God Delusion more than 687,000.

Proffitt concedes that such successes take more effort than they used to: "You have to think more carefully than ever before about every aspect of a book's publication, how it looks, how you communicate its existence." But he insists that the fears for serious books are overblown. "People in the book business are always saying there's a crisis and we're going to hell in a handbasket."

On that point, at least, Proffitt has it over the doom-mongers. Book publishing is a melodramatic business: unpredictable, characterised by very public successes and failures, with even its seasoned practitioners consequently subject to sudden mood swings. In 1934, Geoffrey Faber, one of the founders of Faber, wrote despairingly: "The market is glutted. General publishing is therefore fast degenerating into a gambling competition for potential bestsellers." In 1979, the senior American publisher Jonathan Galassi warned that serious books were becoming "high-risk ventures for modern publishers who have inherited the overhead[s] of big business along with its management techniques and managers, and who have become increasingly reluctant to invest even very modest sums in projects which promise little in the way of immediate return". Non-academic publishing, he went on, may become "nothing more than the calculated marketing of slickly packaged materials, another 'leisure-time' industry ... unconnected to ... the intellectual and cultural roots of the society it purports to serve".

Today, such prophecies look both uncannily accurate and too apocalyptic. On the ground floor of the flagship branch of Waterstone's in Piccadilly, central London, the categories of goods on sale are listed on the wall as follows: "Bestsellers. Gift Wrap & Cards. Humour. Indoor Games. London. Magazines. Maps. Travel. Travel Literature." A huge, mural-sized poster advertising an in-store event by the entertainer Rolf Harris dominates the main entrance. Where books are on display, many of them are in the three-for-two promotions that, once restricted to deals on baby wipes or orange juice in supermarkets, have become central over the last decade to the operation of British chain bookshops.

But there is still serious non-fiction here and there. In recent weeks, there was a place in the front window for Iain Sinclair's chewy new history of Hackney, and a prominent table of thoughtful titles picked by Nick Hornby, including What Good Are the Arts? by the academic and critic John Carey. Esoteric history, the extended critical essay - both titles come from genres whose commercial prospects have been repeatedly written off.

And yet, something is undeniably shifting in the climate for serious books. You can see it in the Piccadilly Waterstone's and in the other cavernous chain bookshops along nearby Charing Cross Road, the traditional heart of British bookselling. Carpets and shop-fittings are worn. Shelves are half-filled. Customers are sparse, even on a weekday lunchtime. There is less bustle, less atmosphere, in these shops than when they opened a decade ago. There is a sense that good times have come and gone.

Good times have never been the norm for serious books in Britain. "From its very beginning, the publishing industry relied on books which could command a large market," writes John Feather in his still-relevant 1988 study A History of British Publishing. "The economics of production ... meant that it could not do otherwise." The 19th century brought mass literacy and a countrywide network of bookshops and publishers, but the business became ever more narrowly focused on producing commercial titles and selling them at a discount. "Booksellers could only afford to stock the most popular and fast-selling," writes Feather, "and had no space on the shelves for ... perhaps more worthwhile works which would sell more slowly and in smaller numbers. The bookshops ... were being squeezed out of business as a market mechanism for serious literature." By the 1890s, so many bookshops had closed and serious books were becoming so marginalised that a consensus formed that book discounting should be banned: the net book agreement was the result, and came into force on 1 January 1900.

For the next three-quarters of a century, British publishing and bookselling were diverse, slow-changing enterprises, relatively tolerant of high-minded books and "ramshackle to the cold business eye", as John Sutherland put it in a 1978 study of the book business. The arrival of state funding for libraries further softened the environment for cerebral writers. In 1965, Britons borrowed 10 books for every one they bought. "With any serious non-fiction book," says Clare Alexander, who entered publishing in 1973, "you knew you could sell 1,500 to the libraries." This enabled mainstream publishers to profitably put out books whose high-street sales were barely in the hundreds - and whose content can seem startlingly uncommercial by current standards. In 1975 Jonathan Cape published Leninism Under Lenin, a labyrinthine theoretical work by the Belgian Marxist historian Marcel Liebman. An equivalent work nowadays would be issued in Britain by a small academic press.

Yet the old British book business had its downsides. Bookshops were often dusty places, glacial in their book-ordering processes and off-putting to young or less educated customers. Even the influence of libraries, Sutherland argues, was not always benign: their preference for books that could be read in a fortnight, the standard lending time, helped keep postwar British writing terse and cramped.

But the tweediness of it all can be overstated. In 1935, in the middle of the depression, Penguin introduced the mass-market paperback to Britain, sold at first through branches of the brash American-owned chainstore Woolworth's, prefiguring today's supermarket book trade. Penguins were initially criticised as crassly commercial and middlebrow, yet soon became seen as a new, accessible form of quality publishing. In the late 1940s, the Better Books chain pioneered the idea of the bookshop as a bright and appealing space, "a social centre with a coffee bar, poetry readings and other literary events", notes Randall Stevenson in The Oxford English Literary History. Meanwhile, wider social changes, such as the growth of higher education, and technical bookselling ones - the introduction of "tele-ordering" in 1979, allowing books to be ordered in days rather than weeks or months - gradually created a bigger potential market for serious titles.

In the 1980s, that market finally materialised. Tim Waterstone was an unemployed man with six children who had already been a tea broker in India, a manager for a brewer and, most recently, the overseer of a disastrous attempt by WH Smith to set up a subsidiary in America, for which he had been fired. In 1982, he used his severance pay and money from friends and relations and the NatWest bank - the latter through the Conservative government's loan guarantee scheme for entrepreneurs - to set up a bookshop in a prosperous part of west London, which was relatively unaffected by the early 1980s recession. The rest of the Waterstone's story has long become bookselling legend: the new shop's daringly deep and heavyweight stock; its unfusty decor and helpful staff; its armchairs for browsers and friendly opening hours. For the next decade and a half, Waterstone's added branches, evangelised for "serious culture", as Waterstone characterised his books, and made literary biography and enigmatic travel writing seem viable high-street products.

New readerships appeared. "In the 90s," says Sutherland, "just before the pension penalty for early retirement came in, there was a mass retirement of teachers. Suddenly, there was a vast number of extremely competent readers who had a lot of time."

The book boom, too, was part of a broader cultural surge that produced the Independent, Channel 4 and scores of new television production companies. The free-market country being created by Margaret Thatcher, however much liberal writers might deplore its harshness, was opening up new possibilities for serious culture, while the non-market mechanisms that protected it, such as the net book agreement, remained largely in place.

During the 1990s this happy equilibrium between commerce and art in the British book business began to break down. By 1989, Waterstone's, like many new 1980s businesses, found that it had expanded too fast and borrowed too much money. Waterstone sold his bookshops to his old corporate foe, WH Smith. Gradually, over the next dozen years, the shops began to stock a narrower range of books.

To a certain extent, Waterstone's was responding to wider developments. Bestseller lists had been published in Britain since 1974, but it was only in the 1990s, with the belated installation of electronic point-of-sale technology in most bookshops, that accurate sales figures for every title became available. Meanwhile, British publishing, which had seen many of its old independent firms taken over by big corporations in the 1980s, had become much more business-oriented. By the mid-1990s, Neil Belton says, "a lot of bigger publishers were ambivalent about the net book agreement". Together with many booksellers, they thought that the freedom to discount titles would improve their profits. In 1995, the agreement in effect collapsed.

That same year, Amazon started selling books via the internet - making even the most rarefied titles universally available, at least in theory, but also offering books at deep discounts and beginning slowly to drain bookshops of their customers. By 2000, British bookselling was beginning to resemble its 19th-century self again: highly competitive, more interested in clever price promotions than clever books. That summer, Waterstone's sacked the manager of its popular Manchester branch, Robert Topping, for refusing to narrow his eclectic stock in favour of bestsellers.

Nine years on, Topping is still in bookselling. He owns and runs two shops, one in Ely in Cambridgeshire and one in Bath. If you like serious books but usually go to the chains, visiting a Topping shop is initially a shock, but then deeply reassuring. "Topping & Company Booksellers of Ely" says the archly old-fashioned sign above the door. Inside, the front of the shop is full of solid biographies, footnoted histories, books on classical music, painting and religion. There are no price promotions, no author posters, no prominently displayed celebrity titles; just plain but expensive-looking wooden bookshelves, neatly jammed with books from the floor to the ceiling and covering every wall through three storeys of small rooms. If you are looking for a John Banville novel, there are seven titles, rather than the couple you usually find in a modern bookshop. If you like the late JG Ballard, there is his genial recent autobiography but also The Atrocity Exhibition (1970). There are newspaper reviews carefully cut and mounted on a pinboard.

Topping is a vicar-ish man in cords and spectacles, but he is not some airy nostalgic. "In the age of the internet, you have to have as wide a stock as you can," he says. "I have to pay the bills myself, so I can see if it works." The computerisation of bookselling, he goes on, ought to help rather than harm serious books: "Distribution is faster, ordering is faster, so a shop can buy single copies and then restock." Is there any sort of serious book that he does think is becoming obsolete? He looks blank for a few moments. "There is this thought that the three-volume life doesn't work these days ..." He pauses again. "Does it work? I suppose not ... But if you believe enough, you can make a book sell."

His shop is busy. Topping chose its site carefully: Ely is a pretty place full of tourists and retired academics, 20 miles from the nearest chain bookshop. But he thinks location is not central to selling serious books: "My hunch is you could do this anywhere."

Belton agrees. "I don't see any evidence that readers are unwilling to grapple with serious ideas in book form, as long as the book is readable. You always hear nonsensical things: 'People aren't interested in science books any more. People aren't interested in history.' That's so wrong. There will always be new subjects, new syntheses, in every field." Peter Straus adds: "In the early 90s, people said political memoirs didn't sell, and then there was Alan Clark's diaries. Anything's possible if the book's good enough." Clare Alexander says: "I don't think the readers of serious books have disappeared. What's happened is a breakdown in delivery. From October onwards - a quarter of the year - bookshop budgets are absorbed by celebrity books."

Some aspects of the life of a serious book have probably changed for good. Anyone who has been writing them for mainstream publishers in the last decade, as I have, will have sensed the new pressures and opportunities: to get your book into the shops at the optimum time of year, to be realistic about its commercial prospects, to promote it through the ever-multiplying media. Randall Stevenson, who is the head of English literature at Edinburgh University, is an advocate of picking your way through dense postmodern texts, yet he senses a new deference in non-fiction to impatient readers. "I read popular science, and it drives me round the bend, all the attention-grabbing and cute fact boxes." The packaging and titles of serious books may also sugar their contents a little more than they used to. Colin Robinson says: "There's quite a retro quality to even the most serious books now. Novels with an old-fashioned soldier on the cover. Books called The Lost City of X."

The internet could have something to do with this. The American scientist Maryanne Wolf, in her book Proust and the Squid: The Story and Science of the Reading Brain (2007), shows that how we read, and what we are easily able and therefore willing to read, is not set, but depends on the kind of texts we are used to interpreting. "I do wonder," she writes, "whether typical young readers view the analysis of text and the search for deeper levels of meaning as more and more anachronistic because they are so accustomed to the immediacy ... of on-screen information."

This may be too pessimistic. Research into the internet's effect on reading is in its infancy. Despite decades of predictions to the contrary, the appetite for demanding non-fiction has survived the advent of newspapers, radio and television - and, in Britain, a popular culture with a particular ability to absorb talent and themes that in other countries would be channelled into grand state-of-the-nation volumes. In fact, many publishers think the noise and immediacy of the web will make slow, quiet immersion in a book seem more, not less, appealing. And books, unlike most digital media, are not directly dependent on recession-affected advertising revenues.

Other economic and social trends remain favourable: the growth of higher education; the proliferation of literary festivals; the falling costs of book production. Tellingly, more people than ever are writing books. "People are infatuated with the romance of writing," says Sutherland. "I can't tell you how many students of mine I recommend an agent to." Dying artforms tend not to attract so many new practitioners.

The crisis in serious non-fiction has probably been overdone. There is a crisis in British bookselling, thanks to the internet, the recession and the particular competitiveness of the British high street - Alexander cites the ever-increasing rents for retail premises. Some non-fiction genres, such as literary biography, are in decline, at least for now. But other serious genres, such as economics and nature writing, are on the rise. Most types of book go through these cycles of boom and bust. In unsettling times, books that try to explain the world may flourish.

In truth, it is too early to tell: serious non-fiction takes time to research and write and sell. But in the meantime, it may be a good idea for authors of such titles to be realistic about their place in the economic order. As John Feather writes in his history of British publishing, before Waterstone's, before agents and advances, before the invention of the modern book business: "The medieval author worked for himself, for God or for a patron, or indeed for all three." I'm not sure that career path would be so popular now.

I tell myself and others I would like to be a full-time writer. However, I fear if I was to become one, I would get detached from the 'real world' and quite possibly end up like Martin Amis. That is a fate I would wish on few!

'I've read the new Martin Amis.' 'Neither have I.' (to use a Peter Cook-ism)

I think writers, like Denis Healey said of politicians, need a 'hinterland' outside their profession, or they get trapped in their own bubble world. The thought of having to write, as opposed to wanting to write, does not appeal to me much (unless the money was really good!). So I will keep writing, when I feel like it, and one day I might just hit the jackpot.

PS There is another short-story competition at work. I've submitted something, which I will publish here once the results are out, towards the end of the month.