\documentclass{beamer}

\mode<presentation> {
  \usetheme{Darmstadt}
  \setbeamercovered{transparent}
}

\usepackage[spanish]{babel}
\usepackage[latin1]{inputenc}
\usepackage{pdfpages}
\usepackage{alltt}
\usepackage{verbatim}
\usepackage{hyperref}

\title[integrating perl in distros]
      {Integrating Perl in a wider distribution: The Debian pkg-perl group}

\author[Gunnar Wolf]{Gunnar Wolf --- {\tt gwolf@debian.org}\\
\url{http://www.gwolf.org/soft/debian_sysadmin}}

\institute[IIEc - UNAM; Debian]
{Instituto de Investigaciones Econ\'omicas, UNAM\\
Desarrollador del proyecto Debian}

\date[YAPC::EU 2007]{
  YAPC::Europe\\
  August 28-30, 2007
}

\pgfdeclareimage[height=2cm]{debian-logo}{debian-swirl}
\logo{\pgfuseimage{debian-logo}}

\AtBeginSection[]
{
  \begin{frame}<beamer>
    \frametitle{Contents}
    \tableofcontents[currentsection]
  \end{frame}
}

% If you wish to uncover everything in a step-wise fashion, uncomment
% the following command: 
%\beamerdefaultoverlayspecification{<+->}

\begin{document}

\begin{frame}
  \titlepage
\end{frame}

%% \begin{frame}
%%   \tableofcontents
%% \end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Who and why?} 

\begin{frame}
  \frametitle {Who am I?}
  \begin{itemize}
    \item Perl programmer for almost ten years (although by far not
      yet an expert)
    \item I presented talks both at YAPC::NA 2001 and YAPC::EU 2002
    \item I currently maintain two (simple, small, low-maintenance)
      CPAN packages
    \item Involved in Debian since 2001; Debian Developer since 2003. 
    \item Member of the Debian {\it pkg-perl} group since its
      formation, recently joined {\it pkg-ruby-extras} 
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {Why am I presenting this? And why here?}
  \begin{itemize}
    \item Programmers want their work to be useful --- Or at least, used
    \item Free Software is one of Perl's most successful {\it
      ecological niches}
    \item The best way for my modules to reach the final user will
      probably be via a {\it Free Software distribution}
    \item Given the way the Perl community works... I expect this talk
      to be like {\it preaching to the choir}
      \begin{itemize}
        \item Perl has a mature, Unix-culture-compatible culture
        \item Many of the tools pkg-perl uses actually derive from
          prominent Perl mongers
      \end{itemize}
    \item I expect to share this with you to better work together, to
      input this experience to other packaging groups in Debian, and
      probably to Perl packaging teams in other distributions
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {Are we really that similar?}
  There might be some sociological reasons for this --- That will be
  up to somebody else to research into. I have at least noticed
  that both communities:
  \begin{itemize}
    \item Value documentation, and require all packages/modules to be
      at least basically documented
    \item Value portability and correctness, even for {\it fringe
      architectures}
    \item Principle of the least surprise; comparatively slow pace of
      development (ensuring not to break the existing user
      base). Things {\it do} change, but changes are gradual and
      well-thought. Both communities have been (wrongly?) blamed for
      stagnating. 
    \item Thousands of packages/modules available in the
      repositories. Many of them are already in a {\it mature} stage,
      with very low change volumes, although very seldom abandoned.
  \end{itemize}
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{CPAN background} 

\begin{frame}
  \frametitle {CPAN background} One of Perl's longest {\it standing
    selling} points is CPAN. It is among the largest repository of
  this kind in the Free Software world, and an immense source of
  resources for its programmers.
\end{frame}

\begin{frame}
  \frametitle {In the beginning, there was CPAN, and it was good}
  \begin{itemize}
    \item The CPAN was born in October 1995
    \item As of May 2007, it hosts 11577 modules from 5856 distinct
      authors
    \item ...And it is a fast-growing list - During 10 days in May, 76
      new modules appeared. Every day, more than 20 uploads were
      received.
    \item CPAN is in no small part responsible for Perl's success,
      giving the users a {\it huge} set of proven, ready libraries to
      work with.
    \item A very important characteristic of CPAN is that it is a {\it
      centralized} repository --- A single entity. We can perform
      different operations (QA, searches, ...) throughout the whole
      module base.
    \item CPAN modules have mostly a homogeneous (well, {\it almost})
      structure and tracking tools, which makes our work to integrate
      them {\it much} easier, technically and socially.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {And then, there is CPAN.pm}
  \begin{itemize}
    \item Having a centralized repository is good, but locally
      managing all the installed modules, versions and dependencies
      quickly becomes {\it hellish}.
    \item In February 1996, Andreas K\"onig creates a client program
      (and Perl module) to automatize CPAN downloads, dependencies and
      versions: {\tt CPAN.pm}. For practical purposes, this solved the
      problem for users.
    \item In 2001, Jos Boumans starts working in CPANPLUS/CPAN++,
      redesigning and modularizing CPAN.pm. It seemed for some time as
      a natural replacement for CPAN - But in the end its ideas ended
      up flowing back, and CPAN still reigns.
    \item They basically cover the {\it Perl community}'s needs on
      managing installation, updates and dependency information.
  \end{itemize}
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Integration with distributions} 

\begin{frame}
  \frametitle {The bigger picture: End users' experiences}
  But all in all, the use CPAN.pm is -for the end-user- quite
  overrated. 
  \begin{itemize}
    \item Users use {\it applications}, do not care about languages!
    \item They don't care about our beautifully crafted tools. Even
      worse, those tools look like hard-to-understand holes in
      reality. 
    \item Every language commumity has its favorite tool: PEAR, Gems,
      Cheese Shop... Does the user really need to know how to use
      them?
    \item Our role, as {\it distribution maintainers}, is to generate
      a unified toolset for our users to manage their {\it whole}
      system coherently --- Say, {\tt aptitude install pkgname} (or
      its point-and-click equivalent).
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {Free Software distributions: Different support tiers}
  \begin{itemize}
    \item The Free Software distributions exist to provide users with
      an integrated, unified system, and take care of passing
      communication between them and upstream authors as needde.
    \item Nowadays Debian is the largest FS distribution (~20,000
      source packages, with strict policies regarding QA and software
      freedom 
    \item Users can file bugs through the Debian BTS, and the {\it
      package maintainer} filters the bug: Is it Debian-specific?
      Pacakging-related? Was it filed for the right package? 
      {\it Should we push it ``upstream''?}
    \item The distribution works as a {\it safety net}, making life
      easier not only for the user, but for the developer as well. Or
      so we wish and even believe ;-)
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {The other way around: Perl supporting the Linux
    distributions (1)}
  But to make this attractive to you: What do Perl Mongers gain from
  us? 
  \begin{itemize}
    \item How can you track your modules across many different
      distributions? 
    \item How can you contact their maintainers {\it everywhere} to
      warn them about a big, nasty API change or reorganization?
    \item What's the best way to notify about disruptive changes {\it
      before} bugs are filed (and users pissed)?
    \item How does each distribution stand up regarding Perl and CPAN
      up-to-dateness and completeness?
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {The other way around: Perl supporting the Linux
    distributions (2)}
  Sadly, right now I don't have the answers to those questions. But
  we are working on them.
  \begin{itemize}
    \item The CPAN ports page's section on Linux gave up ~2004 (and
      IMHO should be just removed)
    \item Gabor Szabo gathered a list of moduless across
      distributions, which is reasonably up to date, but presents
      several interesting problems (even what seems as trivial as 
      {\it what's the current version number of X::Y in Z?})
    \item Maintainers from {\it different distributions} have no
      contact between each other --- So the same problems are likely
      to appear uncoordinately... And swarm the developer. \\
      ...Exactly what we wanted to avoid in the first place
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {A word about propietary systems} 
  We Debianers are known for being passionate about our committment to
  Free Software ideology; some BSDs and Linux distributions are like
  us. Some are known for being just the opposite --- openly
  pragmatic. However, we all at least {\it all} share a Free Software
  background and worldview. But...
  \begin{itemize}
    \item Some people have to (or --gasp-- want to) use a closed
      operating system, such as Windows, MacOS or the historic Unixes
    \item Propietary OSs don't (and probably won't) offer this first
      level of support --- At least, not for their non-core components
    \item If you are in this group... May {\tt \$deity} bless you.
    \item Stick to {\tt CPAN.pm}, and expect no miracles from above.
  \end{itemize}
  ...My deepest sympathy to you guys.
\end{frame}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Debian's pkg-perl group} 

\begin{frame}
  \frametitle {How Debian maintainership works (1)}
  \begin{itemize}
    \item Debian is a {\it social project with a technical
      result}. Around 1000 official developers, hundreds of unofficial
      maintainers, everybody participating strictly on a voluntary
      basis
    \item Each developer is responsable for his own packages. Others
      might {\it touch} them, but it is not considered polite to just
      overtake them.
    \item When you leave the project, you kindly announce your
      packages become {\it orphan}, so that other people take them
      over --- or they can be deleted from the distribution if they
      have no significant user base. 
    \item And yes, we do track our user base! (material for another
      talk, I know)
  \end{itemize}
\end{frame}

\begin{frame}
\frametitle {How Debian maintainership works (2)}
\begin{itemize}
    \item Being a voluntary project, every now and then people leave
      the project... Sometimes without saying. To counter this fact,
      the {\it Debian QA Team} was born. They take care for the
      undermaintained packages, and have some tools for following who
      is active and who is {\it MIA}.
    \item It is also common to seek group maintainership for our
      packages --- That decreases response time for any detected bugs,
      increases the number of people involved in each package, and
      makes it less likely for packages to become orphaned.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {The pkg-perl group}
  \begin{itemize}
    \item In late 2003, the {\tt debian-perl@lists.debian.org} mailing
      list was formed
    \item In early 2004, Joachim Breitner sent out an announcement
      inviting Perl modules' maintainers to join an Alioth (GForge)
      project to start coordinating and avoid duplicating work in
      packaged Perl modules.
    \item {\bf No, CPAN is not --and will not be-- autopackaged}. Only
      the modules we personally use or are requested to (and
      interested in) packaging are.
    \item Our work is carried out on a public SVN repository at
      alioth.debian.org, and coordinated via the {\tt
        debian-perl@lists.perl.org} mailing list and {\tt
        \#debian-perl} channel in {\tt irc.debian.org}.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {Current pkg-perl numbers and tools}
  \begin{itemize}
    \item As of today, the pkg-perl group maintains 370 of the 1303
      Perl module packages in the Debian Unstable branch
    \item Its growth rate is quite strong --- In late May, we were
      handling only 327 packages!
    \item Formally we have almost 50 members registered, but the
      number of {\it active} members is closer to 10-15
    \item We are certainly not bug-free, but --as of July 19, 2007--
      we had only 61 open bugs, two of which were
      release-critical. Eight of those bugs were reported against
      Mime-tools, and most are tagged as ``waiting for upstream to
      decide''. 
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {DEHS: Keeping track of upstream versions}
  \begin{itemize}
  \item One of the most important points in a distribution is
    {\it how to track new upstream versions}. In Debian, we devised a
    mechanism via the magical {\tt debian/rules} file
  \item ...But debian/watch was meant to be run {\it locally} by each
    developer (as it takes {\it stupid} amounts of time to run).
  \item Then came the DEHS ({\tt http://dehs.alioth.debian.org/}) ---
    And Alioth makes all the gruntwork! And it is now integrated into
    all of our maintainer pages ({\tt \scriptsize
      http://qa.debian.org/developer.php?login=gwolf\&comaint=yes})
  \item Weaknesses? Yes, of course. It relies on the ability to check
    periodically over the network. And it often chokes on the silliest
    timeouts. And reports back nasty errors. So we cannot base our
    work only on DEHS.
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle {Evaluating and following the repository as a whole}
  \begin{itemize}
    \item Until some months ago, we went by with only what you saw so
      far. But we have adopted hundreds of modules --- You can only
      expect us to start writing some tools!
    \item Our current scripts were started by Gregor Hermann, and are
      part of our SVN tree. The scripts basically cover:
  \end{itemize}
  \begin{description}
    \item [build\_logs] We periodically build all of our
      packages. This script reports the full logs, sorted by build
      result
    \item [maintainers] Checks our SVN directory to check if any of
      our modules does not have an updated maintainer/uploader
      information
    \item [versions] Compares the versions of the modules in our tree,
      in Debian unstable and in CPAN. Of course, it has the same good
      and bad points of DEHs
    \item [wnpp] Track new requests for packaging by our users,
      orphaned packages, etc.
  \end{description}
\end{frame}

\begin{frame}
  \frametitle {dh-make-perl}
  \begin{itemize}
    \item We will not blindly package all of CPAN... But we want to
      provide our users with tools so they can still benefit from an
      unified packaging system
    \item Back in October 2000, Paolo Molaro made the first upload of
      {\tt dh-make-perl}. Starting in October 2006, the pkg-perl group
      adopted it. It is a program that takes a standard Perl module
      and generates a Debian package for it, including:
      \begin{itemize}
        \item Name and version
        \item Short and long description
        \item Infrastructure needed for building (i.e. modules used
          for tests)
        \item Dependencies for using this module
        \item Information regarding the current base Perl language and
          modules installed
      \end{itemize}
    \item It is messy and not pretty... But it is an invaluable tool :)
  \end{itemize}
\end{frame}

\begin{frame}
  \frametitle{Questions? Comments? Whatever?}
  \vskip 2em
  {\Large Thank you very much!}
  \vskip 1em
  Gunnar Wolf --- {\tt gwolf@gwolf.org}
%%   \url{http://www.gwolf.org/soft/integrating_perl_in_distro}
\end{frame}

\end{document}