Click here to register.

Google Summer of Code Ideas Page

Flat
Google Summer of Code 2008 - post-mortem
User: kmaclean
Date: 3/17/2008 7:34 pm
Views: 4432
Rating: 28

I'm starting this as a post-mortem thread.  A post-mortem is basically an after-the-fact review of the things (e.g. the GSoC application, our ideas page, the VoxForge website) to figure out how to improve things for next time.  Any feedback on what we could have done better with would be greatly appreciated.

I also came across this page on the Google site today that would be helpful for next year's application:

Notes on Organizations Selection Criteria

Organizations for Google Summer of CodeTM are chosen each year from the applicant pool by Chris DiBona, Greg Stein and Leslie Hawthorn.  We don't have any hard and fast rules for selecting organizations - there are many organizations that apply and many more than apply who we'd like to help with their open source efforts.  In the end, we have to make many tough decisions about which organizations to accept.  Here's a bit about our thought process:

1) Have you been in GSoC before and done well?  We want to make sure that all of our students have a good experience, and an established track record of success is a great indicator for the future.  Keep in mind, though, we vary the mix somewhat each year regardless of how well an organization did in the past. We may vary it even more broadly in the future.

2) Do the projects on your ideas list look feasible for student developers?  Is your ideas list thorough and well-organized?  Your ideas list is the first place that student participants are going to look to get information on participating in GSoC, so putting a lot of effort into this list is a good thing(tm).  One thing we noticed and really appreciated this year was how some organizations classified their ideas by easy, medium and difficult, and specifically listed the skills and background required to complete a given task.  It might also be cool to expand on each idea with some places to get started research-wise (pointers to documentation or specific bugs), as well as the impact finishing a given idea will have for the organization.

3) Did it look like you spent a decent amount of time on your application?  Do we have a good sense of who you are and what you intend to accomplish in GSoC after reading it?  Just as with an organization reviewing student applications, a thorough and well-written org application piques our interest.  If it looks like you only spent ten minutes on the application, er, not so much.  Particularly when the application questions were published weeks in advance of the application period. 

4) Do we use your software?  Do we know someone who does, who also raves about it?  If so, your organization looks that much more attractive to us.

5) Do we have a relationship with your developer community?  Has your project been recommended to us by folks we trust?  If you're a known quantity, we're in a better place to make a judgement about whether or not you'll be able to support our students well.  If you're hoping to establish a relationship with us, we're pretty easy to find.  Chat with us at a conference or drop us a mail during one of the slower times for the program.  We would love to hear from you.

6) Will funding your organization have a significant impact on the open source world?  Niche projects with few users and developers are less likely to be accepted.

In addition to all these items, there's also the intangibles.  Sometimes, we just think an idea sounds cool and we want to help. 

A final note, an organization may fulfill some of these criteria and still not be accepted.  We simply can't accept every organization that applies. 

thanks,

Ken 

 

2. Do the projects on your ideas list look feasible for student developers
User: kmaclean
Date: 3/17/2008 7:39 pm
Views: 220
Rating: 24
One thing I am thinking of is that I/we could have worked on #2 (Do the projects on your ideas list look feasible for student developers?) a lot more by classifying the ideas as easy, medium and difficult, and listing the skills and background required to complete a given task, and add research pointers.
5. Do we have a relationship with your developer community
User: kmaclean
Date: 3/17/2008 7:40 pm
Views: 221
Rating: 32
With respect to #5 (Do we have a relationship with your developer
community?),  I've put a reminder in my 'Google  Calendar' to contact Leslie Hawthorn in September (after GSoC winds down) to introduce myself, and the VoxForge project.
Re: Google Summer of Code 2008 - post-mortem
User: kmaclean
Date: 3/17/2008 8:01 pm
Views: 185
Rating: 26
Another link that will prove useful for next year:
Advice for GSoC Mentoring Organizations and Mentors

    Creating Your "Ideas" List

    Choosing tasks to include on your project's ideas list can be challenging: some bugs may have remained open for years because a problem is extremely difficult to solve, some tasks which could bore a more advanced student may prove the perfect training ground for a less advanced student. What steps can you take to make sure your project's Ideas list attracts the right mix of applicants?

    Be Specific. No, Be General.

    Some students will be attracted to ideas that are carefully described and documented by your team. Providing a thorough roadmap for the implementation of the idea helps prospective students judge whether they are a good fit for the program, and whether they feel confident in their ability to actually complete the work.

    The more specifically you describe your ideas before the program begins, the more students will apply with those ideas as their proposals. Sorting through these proposals is difficult because you know where the idea came from, thus the student applicants don't have a chance to show creativity (by thinking of the idea to begin with), and you are left trying to weigh the other factors in their application, like their prior experience or grade point average.

    On the other hand, some students will have their own ideas, or will thrive when given a vague goal and have to provide the details for achieving that goal on their own. It is a disadvantage for these students when you provide super-specific specifications in advance of the program. The applications of these "creative" students are easier to judge. First, they are presenting their own idea. The merit of the idea, as well as how it is presented, speaks loudly for the abilities of the student. Second, if any details are provided as to how the student hopes to implement their idea, you get a good sense of how familiar they already are with your project and the problem space they'll be dealing with.

    The lesson here? To attract the widest variety of students, provide a variety of proposal ideas in different stages of detail. If your project needs a certain feature, and it is crystal clear how you need it implemented, spell it out. If you'd like to see general development in a certain direction, leave the door open for the students' proposals to be creative and encourage them to fill in the details themselves.

    At the 2007 mentor summit Leslie Hawthorn shared this important factoid:

        The most successful projects are the ones that are proposed by the students.

    Scope and Scale

    It is in everybody's best interest that your students complete their projects with flying colors. A small amount of useful code is worth more than a mountain of almost finished code. A happy student at the end of the program is more likely to keep working on his/her project than a frustrated one. Choose the scope and scale of your projects to reflect the following constraints:

        * The brief duration of the program.
        * The initial time that will be spent managing infrastructure and getting things set up.
        * The fact that most students need the first month just to get their bearings and actually start producing code.
        * The fact that any project that is worthwhile has inherent difficulties and real intellectual challenges, some of which won't be anticipated or predicted at the onset of the project.

    Therefore, whether the idea for the project comes from you or from your student, be ruthless in scaling it back until it really fits within the time constraints. If all goes well, you and the student will have a lot of time after the program to implement phase 2 of the project, and add all those features that would be "nice to have".

    Generating Ideas

    Google Summer of Code can be useful to your project in ways that are not directly tied to gaining code and new contributors. For example, most projects have a portion of their community that is brimming with ideas and feature requests, but is not directly able to initiate the implementation of their ideas (ie they can't code). These are the people you want to engage when creating proposals for students to work on. Use your project's website and mailing list to collect ideas from all segments of your community, and keep them up to date with the status of the projects as the program progresses. Your students and your community will benefit from this.

    Be Accessible

    A successful SoC student will have good communication skills. This is a simple fact of the nature of distributed projects (such as most open source projects). The people involved must regularly overcome the barriers of distance, language, culture and time zones in order to come to agreements and exchange knowledge.

    By making yourself available to discuss the proposed projects in advance, you are inviting the good communicators to reach out to you and demonstrate that they are suited for working on a virtual team where independence and cooperation must coexist. They will ask for suggestions on their proposals, bounce their ideas off you to see if they fit your project, and reveal aspects of their personality that will be useful to you when evaluating their applications later.

    In most mentor organisations there will be more applications than slots available. This was certainly true at the ASF where the sheer number of projects we have means that we have had at least four times as many proposals as funded slots in previous years. In order to select the best projects committers interested in the GSoC programme have reviewed all the students applications, so the quality of the proposal was important. However, we added considerable weight to those proposals that had entered into a dialogue with the project community already. Just being smart is not good enough in open source, the ability to ask for and accept guidance is even more important.

    
Re: Google Summer of Code 2008 - post-mortem
User: nsh
Date: 3/18/2008 3:42 am
Views: 180
Rating: 29

I'd like to comment on this a bit

First of all, Google is not the only organization we can raise funds from. Moreover, SoC is not about getting work done but more about teaching the students, I'm not sure we'd like to do this.

There are already some organization interested in speech technologies and willing to pay money for them. We can fill this niche but we need one major thing we miss now - end-user product. It's really hard to promote/sell something abstracted like speech corpora. Dialog managers, IVR, speech search services are what users really need. Google collects databases but it's not the primary product they promote, they sell services. The database is only a side effect. 

So we have a lot of opportunities and lot of work to do. Bounties will come soon. 

Re: Google Summer of Code 2008 - post-mortem
User: kmaclean
Date: 3/18/2008 11:29 am
Views: 160
Rating: 22

Hi nsh,

thanks for the comments! 

>First of all, Google is not the only organization we can raise funds from. 

Agree. 

>Moreover, SoC is not about getting work done but more about teaching the

>students, I'm not sure we'd like to do this.

I realized this more as I watched the IRC chat discussions on the #GSOC channel.   Here are some of the more important points I gleaned from the discussions with projects who were not accepted:

  •  the point is the students after all, much as some of us smaller projects would like it to be about helping us :)
  • ideas list
    •  we need you tell students what you want, why you want it done, what they need to know to accomplish it
  •  even if stuff is in our issue tracker, we rewrite it for SoC so its clearer for someone who doesn't know what a FooBidget is and where it fits into the picture...
  •  ideas list is underspecified - no links to further resources, no easy medium hard designations, it's pretty spartan
  •  "needs more detail, and needs more instructions what a successful application will look like"?
  • this is a laundry list of wants for your project, it is not tailored to students at all
  •  SoC is 3 months. So all the projects need to fit into a 3 month slot. for someone who doesn't know your project
  •  your ideas list is not targeted - it is a list of things you want with no further supplementary information: no what languages you need to know, no prerequisites, no difficulty slotting
  • good ideas pages: GIT
  • IRC: be involved on [the Google] IRC as part of submitting your proposal is really important; makes your name and idea stick out in peoples' minds.
*If* Google SOC is what we want to do, then that is what we need to do for next year.  To this end, we could use some project management techniques to create estimates.  The Mifos project uses an Agile approach (Planning Poker) for estimating durations that might be helpful.

>There are already some organization interested in speech technologies and

>willing to pay money for them. We can fill this niche but we need one major

>thing we miss now - end-user product. It's really hard to promote/sell

>something abstracted like speech corpora. Dialog managers, IVR, speech

>search services are what users really need. Google collects databases but

>it's not the primary product they promote, they sell services. The database

>is only a side effect. 

VoxForge's general goal is to advance FOSS speech recognition overall, and specifically to create a commercial equivalent speech corpus.  One way to do this is organically - which is what we are doing.  This will take time.  To help speed things up, we can look outside the core group for help. 

GSoC is one approach, though, as you said, it is student focused.

Providing services/products to organizations who are willing to pay for speech recognition products or services is another approach.  Creating a product that would catch users' attention would go a long way to promoting VoxForge, and encouraging people to submit additional speech.  I am not sure that a desktop command and control dialog manager would necessarily be the best way to go.  My sense, and I may be wrong, is that telephony is our best bet, and it may be by offering turn-key open source speech recognition solution.  There is no fully open-source turn-key solution out there.  Some were started, but most stalled at the speech recognition portion of it.  We could take something like jvoicexml or VoiceGlue (or partner with them), and create a fully working downloadable rpm package, and use this to promote VoxForge, and get people to contribute speech.

Is this what you were thinking about? 

Ken 

Re: Google Summer of Code 2008 - post-mortem
User: nsh
Date: 3/18/2008 2:30 pm
Views: 1585
Rating: 28

> Is this what you were thinking about?

The same things, nothing more to add :) Except if you are hanging on irc, join #cmusphinx on freenode, I always have a few minor requests to put.

 

PreviousNext