Consider the following simple file:

% -*-latex-*-
% Document name: /tmp/foo.ltx
% Creator: Nelson H. F. Beebe [[log in to unmask]]
% Creation Date: Wed Oct 15 12:13:15 1997
\documentclass[]{article}

\documentclass[]{article}

\usepackage[]{amsmath}

\newenvironment{fancyequation}
               {\hrule \begin{equation}}
               {\end{equation}\hrule}

\begin{document}

  \begin{fancyequation}
 a = b
  \end{fancyequation}

\end{document}

When I run it on my local TeX system, or on the TeXlive one, I get an error:

$ /cdrom/texlive/bin/sparc-solaris2.5/latex foo.ltx
This is TeX, Version 3.14159 (Web2c 7.0)
(foo.ltx
LaTeX2e <1996/12/01> patch level 1
Babel <v3.6h> and hyphenation patterns for american, english, french, dutch,
ge
rman, loaded.
(/cdrom/texlive/share/texmf/tex/latex/base/article.cls
Document Class: article 1996/10/31 v1.3u Standard LaTeX document class
(/cdrom/texlive/share/texmf/tex/latex/base/size10.clo))
(/cdrom/texlive/share/texmf/tex/latex/amslatex/amsmath.sty
(/cdrom/texlive/share/texmf/tex/latex/amslatex/amstext.sty
(/cdrom/texlive/share/texmf/tex/latex/amslatex/amsgen.sty))
(/cdrom/texlive/share/texmf/tex/latex/amslatex/amsbsy.sty)
(/cdrom/texlive/share/texmf/tex/latex/amslatex/amsopn.sty)) (foo.aux)
Runaway argument?

! Paragraph ended before \equation was complete.
<to be read again>
                   \par
l.18

If I comment out the \usepackage command, it works as expected:

$ /cdrom/texlive/bin/sparc-solaris2.5/latex foo.ltx
This is TeX, Version 3.14159 (Web2c 7.0)
(foo.ltx
LaTeX2e <1996/12/01> patch level 1
Babel <v3.6h> and hyphenation patterns for american, english, french, dutch,
ge
rman, loaded.
(/cdrom/texlive/share/texmf/tex/latex/base/article.cls
Document Class: article 1996/10/31 v1.3u Standard LaTeX document class
(/cdrom/texlive/share/texmf/tex/latex/base/size10.clo)) (foo.aux) [1]
(foo.aux)
 )
Output written on foo.dvi (1 page, 312 bytes).
Transcript written on foo.log.

Is this exposing a bug in the implementation of \end{equation} in
the amsmath package?

Ordinarily, there should be no problem in using an \end{environment}
inside the definition of another macro.

----------------------------------------------------------------------------
- Nelson H. F. Beebe                  Tel: +1 801 581 5254                 -
- Center for Scientific Computing     FAX: +1 801 581 4148                 -
- University of Utah                  Internet e-mail: [log in to unmask] -
- Department of Mathematics, 105 JWB                   [log in to unmask]       -
- 155 S 1400 E RM 233                                  [log in to unmask]      -
- Salt Lake City, UT 84112-0090, USA  URL: http://www.math.utah.edu/~beebe -
----------------------------------------------------------------------------