(***********************************************************************

                    Mathematica-Compatible Notebook

This notebook can be used on any computer system with Mathematica 4.0,
MathReader 4.0, or any compatible application. The data for the notebook 
starts with the line containing stars above.

To get the notebook into a Mathematica-compatible application, do one of 
the following:

* Save the data starting with the line of stars above into a file
  with a name ending in .nb, then open the file inside the application;

* Copy the data starting with the line of stars above to the
  clipboard, then use the Paste menu command inside the application.

Data for notebooks contains only printable 7-bit ASCII and can be
sent directly in email or through ftp in text mode.  Newlines can be
CR, LF or CRLF (Unix, Macintosh or MS-DOS style).

NOTE: If you modify the data for this notebook not in a Mathematica-
compatible application, you must delete the line below containing the 
word CacheID, otherwise Mathematica-compatible applications may try to 
use invalid cache data.

For more information on notebooks and Mathematica-compatible 
applications, contact Wolfram Research:
  web: http://www.wolfram.com
  email: info@wolfram.com
  phone: +1-217-398-0700 (U.S.)

Notebook reader applications are available free of charge from 
Wolfram Research.
***********************************************************************)

(*CacheID: 232*)


(*NotebookFileLineBreakTest
NotebookFileLineBreakTest*)
(*NotebookOptionsPosition[      8500,        253]*)
(*NotebookOutlinePosition[      9156,        276]*)
(*  CellTagsIndexPosition[      9112,        272]*)
(*WindowFrame->Normal*)



Notebook[{

Cell[CellGroupData[{
Cell["Convolution and padding", "Subtitle"],

Cell[TextData[{
  "Before I try to explain the arguments of  ",
  ButtonBox["ListConvolve",
    ButtonStyle->"RefGuideLink"],
  ", I will show you a simple function that the operations are based on."
}], "Text"],

Cell[BoxData[
    \(BasicConvolve[kernel_List, \ 
        list_List]\  := \[IndentingNewLine]Module[{rev\  = \ 
            Reverse[kernel], \ 
          n\  = \ Length[kernel]\  - \ 1}, \[IndentingNewLine]Table[
          Take[list, {i, i + n\ }] . rev, \ {i, 
            Length[list]\  - \ n}]]\)], "Input"],

Cell[BoxData[{
    \(\(Clear[a, b];\)\), "\[IndentingNewLine]", 
    \(\(A\  = \ Table[a\_i, {i, 4}];\)\), "\[IndentingNewLine]", 
    \(\(B\  = \ Table[b\_j, {j, 5}];\)\)}], "Input"],

Cell[BoxData[
    \(BasicConvolve[A, B]\)], "Input"],

Cell[BoxData[
    \(ListConvolve[A, B]\)], "Input"],

Cell["\<\
In many cases, you want to do something special with the data at the edges.  \
For multiplication, you want to get the products of the subparts.  In cyclic \
convolution, you want to wrap the ends around periodically.  In either case, \
the situation can still be done with BasicConvolve, but using padding.\
\>", "Text"],

Cell[TextData[{
  "Functions to do ",
  ButtonBox["padding ",
    ButtonData:>{"1.8.6", "6.13"},
    ButtonStyle->"MainBookLink"],
  "of tensors have been added:"
}], "Text"],

Cell[BoxData[
    \(Bpad\  = \ 
      PadLeft[B, Length[B]\  + \ 2\ Length[A\[IndentingNewLine]] - 2, \ 0, \ 
        Length[A] - 1]\)], "Input"],

Cell["Now, for the acyclic convolution, I can use", "Text"],

Cell[BoxData[
    \(TableForm[BasicConvolve[A, Bpad]]\)], "Input"],

Cell["\<\
The arguments of ListConvolve allow for padding.  The third argument \
specifies (in a way) how far the kernel should \"overhang\" the list at the \
either end of the convolution and the fourth argument specifies what to use \
for padding.\
\>", "Text"],

Cell[BoxData[
    \(%\  === \ ListConvolve[A, B, {1, \(-1\)}, 0]\)], "Input"],

Cell["\<\
The first 1 means the first element of the kernel should line up with the \
first element of the list at the beginning, and that the last (-1) element of \
the kernel should line up the the last element of the list.\
\>", "Text"],

Cell["The cyclic convolution can simply be handled by ", "Text"],

Cell[BoxData[
    \(Bpad\  = \ 
      PadRight[B, Length[B]\  + \ Length[A\[IndentingNewLine]] - 1, \ 
        B]\)], "Input"],

Cell["Now, I can use ", "Text"],

Cell[BoxData[
    \(TableForm[BasicConvolve[A, Bpad]]\)], "Input"],

Cell["\<\
If the padding argument is eliminated, it is taken to be the \"list\" or \
second argument.\
\>", "Text"],

Cell[BoxData[
    \(%\  === \ ListConvolve[A, B, {\(-1\), \(-1\)}]\)], "Input"],

Cell["Note that ", "Text"],

Cell[BoxData[
    \(BasicConvolve[A, 
        PadLeft[B, Length[B]\  + \ Length[A]\  - \ 1, \ B]]\  === \ 
      ListConvolve[A, B, {1, 1}]\)], "Input"],

Cell["\<\
An old command which has been extended to alow padding is Partition\
\>", "Text"],

Cell[BoxData[
    \(Partition[Range[7], 2]\)], "Input"],

Cell[BoxData[
    \(Partition[Range[7], 2, 2, {1, 1}]\)], "Input"],

Cell["\<\
The end conditions are similar to ListConvolve: you can think of the kernel \
analogue as being of length 2.  In this example, the analogy is closer\
\>", "Text"],

Cell[BoxData[
    \(Partition[Range[7], 2, 1, {1, \(-5\)}]\)], "Input"],

Cell["\<\
The default is for cyclic padding, but any list (or single value) can be used\
\
\>", "Text"],

Cell[BoxData[
    \(Partition[Range[7], 2, 1, {1, \(-5\)}, A]\)], "Input"],

Cell[BoxData[
    \(Partition[Range[7], 2, 1, {1, \(-5\)}, {Sequence[]}]\)], "Input"],

Cell["\<\
Now back to convolution and multiplication.  Suppose that we have two lists \
of random digits.  Compare the timings for ListConvolve and BasicConvolve\
\>", "Text"],

Cell[BoxData[
    \(Do[\[IndentingNewLine]n\  = \ 10^k; \[IndentingNewLine]m\  = \ 
        10000/n; \[IndentingNewLine]A\  = 
        Table[Random[Integer, {0, 9}], {n}]; \[IndentingNewLine]B\  = \ 
        Table[Random[
            Integer, {0, 
              9}], {n}]; \[IndentingNewLine]tlc\  = \ \(Timing[\(Do[
                ListConvolve[A, B, {1, \(-1\)}, 
                  0], {m}];\)]\)[\([1]\)]; \[IndentingNewLine]tbc\  = \ \
\(Timing[\(Do[BasicConvolve[A, \ 
                  PadLeft[B, 
                    Length[B]\  + \ 2\ Length[A\[IndentingNewLine]] - 
                      2]], {m}];\)]\)[\([1]\)]; \[IndentingNewLine]Print[{n, 
          tlc, tbc, tbc/tlc}], \[IndentingNewLine]{k, 1, 4}]\)], "Input"],

Cell["\<\
So ListConvolve must be doing something other than simply streamlining the \
operations.\
\>", "Text"],

Cell["\<\
In fact it is taking advantage of the discrete convolution theorem and the \
compuational efficiency of the fast Fourier transform.  \
\>", "Text"],

Cell["\<\
Lets go back to the simple multiplcation example.  An alternate way to do the \
acyclic convolution would be:\
\>", "Text"],

Cell[BoxData[
    \(FourierAcyclicConvolve[A_, \ 
        B_]\  := \ \[IndentingNewLine]Module[{len, \ Apad, \ 
          Bpad}, \[IndentingNewLine]len\  = \ 
          Length[A]\  + \ Length[B]\  - 1; \[IndentingNewLine]Apad\  = \ 
          PadLeft[A, len\ , 0, 
            len\  - \ Length[A]]; \[IndentingNewLine]Bpad\  = \ 
          PadLeft[B, len, 0, len\  - \ Length[B]]; \[IndentingNewLine]Round[
          InverseFourier[
            Fourier[Apad, \ FourierParameters \[Rule] {1, 1}]\ Fourier[
                Bpad, \ FourierParameters \[Rule] {1, 1}], \ 
            FourierParameters \[Rule] {1, 1}]]]\)], "Input"],

Cell[BoxData[
    \(FourierAcyclicConvolve[{1, 2, 3, 4}, {5, 6, 7, 8}]\)], "Input"],

Cell[BoxData[
    \(Do[\[IndentingNewLine]n\  = \ 10^k; \[IndentingNewLine]m\  = \ 
        10000/n; \[IndentingNewLine]A\  = 
        Table[Random[Integer, {0, 9}], {n}]; \[IndentingNewLine]B\  = \ 
        Table[Random[
            Integer, {0, 
              9}], {n}]; \[IndentingNewLine]tlc\  = \ \(Timing[\(Do[
                ListConvolve[A, B, {1, \(-1\)}, 
                  0], {m}];\)]\)[\([1]\)]; \[IndentingNewLine]tbc\  = \ \
\(Timing[\(Do[FourierAcyclicConvolve[A, \ 
                  B], {m}];\)]\)[\([1]\)]; \[IndentingNewLine]Print[{n, tlc, 
          tbc, tbc/tlc}], \[IndentingNewLine]{k, 1, 4}]\)], "Input"],

Cell["\<\
What ListConvolve does is use complexity estimates to decide if an FFT would \
be faster or not.  With integers, it uses techniquest to make sure that \
absolutely no rounding errors are introduced by the FFT, which is done with \
machine numbers.\
\>", "Text"],

Cell["\<\
Multiplication works essentially the same way, but takes into account the \
complexity of the Karatsuba method.\
\>", "Text"]
}, Open  ]]
},
FrontEndVersion->"4.0 for Microsoft Windows",
ScreenRectangle->{{0, 1024}, {0, 695}},
WindowSize->{496, 599},
WindowMargins->{{Automatic, 117}, {Automatic, 30}}
]


(***********************************************************************
Cached data follows.  If you edit this Notebook file directly, not using
Mathematica, you must remove the line containing CacheID at the top of 
the file.  The cache data will then be recreated when you save this file 
from within Mathematica.
***********************************************************************)

(*CellTagsOutline
CellTagsIndex->{}
*)

(*CellTagsIndex
CellTagsIndex->{}
*)

(*NotebookFileOutline
Notebook[{

Cell[CellGroupData[{
Cell[1739, 51, 43, 0, 64, "Subtitle"],
Cell[1785, 53, 211, 5, 52, "Text"],
Cell[1999, 60, 310, 6, 90, "Input"],
Cell[2312, 68, 183, 3, 70, "Input"],
Cell[2498, 73, 52, 1, 30, "Input"],
Cell[2553, 76, 51, 1, 30, "Input"],
Cell[2607, 79, 331, 5, 90, "Text"],
Cell[2941, 86, 174, 6, 33, "Text"],
Cell[3118, 94, 145, 3, 50, "Input"],
Cell[3266, 99, 59, 0, 33, "Text"],
Cell[3328, 101, 66, 1, 30, "Input"],
Cell[3397, 104, 263, 5, 71, "Text"],
Cell[3663, 111, 77, 1, 30, "Input"],
Cell[3743, 114, 239, 4, 71, "Text"],
Cell[3985, 120, 64, 0, 33, "Text"],
Cell[4052, 122, 126, 3, 50, "Input"],
Cell[4181, 127, 31, 0, 33, "Text"],
Cell[4215, 129, 66, 1, 30, "Input"],
Cell[4284, 132, 115, 3, 33, "Text"],
Cell[4402, 137, 79, 1, 30, "Input"],
Cell[4484, 140, 26, 0, 33, "Text"],
Cell[4513, 142, 152, 3, 50, "Input"],
Cell[4668, 147, 91, 2, 33, "Text"],
Cell[4762, 151, 55, 1, 30, "Input"],
Cell[4820, 154, 66, 1, 30, "Input"],
Cell[4889, 157, 172, 3, 52, "Text"],
Cell[5064, 162, 71, 1, 30, "Input"],
Cell[5138, 165, 103, 3, 33, "Text"],
Cell[5244, 170, 74, 1, 30, "Input"],
Cell[5321, 173, 85, 1, 30, "Input"],
Cell[5409, 176, 175, 3, 52, "Text"],
Cell[5587, 181, 726, 13, 310, "Input"],
Cell[6316, 196, 112, 3, 33, "Text"],
Cell[6431, 201, 157, 3, 52, "Text"],
Cell[6591, 206, 133, 3, 52, "Text"],
Cell[6727, 211, 627, 11, 190, "Input"],
Cell[7357, 224, 83, 1, 30, "Input"],
Cell[7443, 227, 629, 11, 230, "Input"],
Cell[8075, 240, 271, 5, 71, "Text"],
Cell[8349, 247, 135, 3, 52, "Text"]
}, Open  ]]
}
]
*)




(***********************************************************************
End of Mathematica Notebook file.
***********************************************************************)