I think your objective of using Mathematica to write literate technical documents is a very proper and useful approach. I can be done but there is probably a lot to be learned because an active, dynamic document has much more potential capability for development and communication than a static written document.
Here are some tips:
1) Learn to use the Sectional organization of notebooks.
2) Some of your objectives, such as italicized single character symbols, can be used by using TraditionalForm for Output. (I myself prefer StandardForm because it is not ambiguous but some sophisticated users swear by TraditionalForm.)
3) Mathematica will not automatically use different font styles for different types of variables and I doubt if it would be easy to make it automatic. You can use the SpecialCharacters palette for different font styles and adhere to conventions in your own usage. (Starting variable names with capital letters is generally frowned upon because they may conflict with Mathematica built-in symbols.) Generally I would recommend against being too concerned about variable fonts and meanings. There is a lot to be said for regular keyboard characters and longer descriptive names.
4) It's possible to generate special output formats using MakeBoxes.
5) Mathematica has both a pi symbol and pi as a numeric constant.
5) Style sheets allow you do define different formats for different cell types, including new cell types you might create, but they will not allow different styles for different types of symbols within one cell type.
6) There are a number of techniques for hiding boilerplate specifications and displaying text and output in a smooth manner. It is also fairly easy to generate displays in separate windows so a reader can scroll the notebook while observing the diagram. (One of the advantages of ancient scrolls was that they could be spread out so text and diagrams were both visible. The diagram was not on the opposite side of the page!) One of the nice features of notebooks is that you can bring the material to the reader, where he needs it, without jumping around in the document.
7) It is important to keep the science and mathematics front and center and not impose extra programming system restrictions that become more of a burden than a help.
I sell an Application called Presentations through my web site that has many features that make it easier to write custom graphics, tables and dynamic presentations. It also has many features that help to write notebooks as readable and usable technical documents. There are also essays on Writing Notebooks, Writing Presentations, Writing Packages and Writing Applications. Also many examples and extended examples including longer notebooks in the form of technical documents. You can certainly write nice documents without it, but many people find it useful.
I am very new to Mathematica but see its potential. I'm unsure of how to combine text and Mathematica computations in a notebook so that the final document reads properly, and I haven't found a tutorial or any other help that covers this. A near as I can make out, one writes the document as if it were an ordinary word processing document, interspersing Mathematica calculations and commands, and then one collapses the cells that one does not want to display in the final document. Is this correct?
What I'd like to do is to adopt certain notational conventions and have mathematica deal with them correctly. For example, one set of conventions italicizes names of all variables, uses uppercase letters to name random variables and matrices, but distinguishes random variables from matrices by displaying all matrix names in boldface. A different set of conventions will use italicized uppercase letters to designate points in a diagram or constant parameters in an equation, with matrices now designated as uppercase letters with the dimensions of the matrix as a subscript (e.g., n x m).
Ideally, one would have a mathematical style sheet that makes changing the conventions used in a document. But even if one were to be willing to use a fixed notation in a document (notebook), I'm not sure how it would work. How does one use X to designate a random variable and bold X to designate a matrix? How does one use the Greek letter pi as a variable and have it show up in the document as the Greek letter rather than as the word "pi"?