SlideShare a Scribd company logo
1 of 131
Download to read offline
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




Music Recommendation and Discovery
           Remastered

                                  Tutorial



                        @recsys, 2011
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




                @plamere                          @ocelma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




INTRO CHORUS VERSE BRIDGE OUTRO
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




INTRO CHORUS VERSE BRIDGE OUTRO
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Music Recommendation is important

       How many songs fit in my pocket?




  10 Songs                   1,000 Songs                      10,000,000 Songs
    1979                         2001                               2011
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


What's so special about music?
●   Huge item space
●   Very low cost per item
●   Many item types
●   Low consumption time
●   Very high per-item reuse
●   Highly passionate users
●   Highly contextual usage
●   Consumed in sequences
●   Large personal collections
●   Doesn't require our full attention
●   Highly Social
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


  Music recommendation is broken ...
 If you like Britney Spears
 you might like...




...Report on Pre-War Intelligence

Let's look at some of the
issues ....
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma



    What makes a good music recommendation?
●   Relevance
●   Novelty / Serendipity
●   Transparency / Trust
●   Reach
●   Context
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma



    What makes a good music recommendation?
●   Relevance
●   Novelty / Serendipity
●   Transparency / Trust
●   Reach
●   Context
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma

                Relevance
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma

Relevance – cold start
new or unpopular items
 If you like Gregorian Chants you might like Green Day
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Cold Start – New User - Enrollment
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


      New User – Implicit taste data
The Audioscrobbler
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Relevance – Metadata Mismatches
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Relevance – Metadata Mismatches



                                                                       Why?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Relevance - The grey sheep problem
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Relevance – Cultural Mismatches
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


    What makes a good music recommendation?

●   Relevance
●   Novelty / Serendipity
●   Transparency / Trust
●   Reach
●   Context
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


     Novelty and Serendipity
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma



Popularity Bias - The Harry Potter Effect
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma

...also known as the Coldplay effect
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


    Novelty / Serendipity – the enemy




High stakes competitions focused on relevance can
reduce novelty and serendipity
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


    What makes a good music recommendation?

●   Relevance
●   Novelty / Serendipity
●   Transparency / Trust
●   Reach
●   Context
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma

The Opacity Problem
β€œIf you like NiN you might like Johnny Cash”



                                                                     Why???
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


  Is this a good recommendation?

If you like Norah Jones ...




You might like Ravi Shankar
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


  Is this a good recommendation?

If you like Norah Jones ...




You might like her father, Ravi Shankar
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma



      Brutal Death Metal Quiz


???????




                                                                              Photo cc by Mithrandir3
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma



  Brutal Death Metal Quiz
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


 Hacking the recommender
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


    What makes a good music recommendation?

●   Relevance
●   Novelty / Serendipity
●   Transparency / Trust
●   Reach
●   Context
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma

Help! I’m stuck in the head
The limited reach of music recommendation




                           48% of recommendations
      Popularity




                                                                                   0% of
                                                                              recommendations

                                         52% of recommendations



        83 Artists             6,659 Artists               239,798 Artists

Study by Dr. Oscar Celma - MTG UPF
                                                  Sales Rank
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Music Discovery Challenge
Personal discovery a challenge too

                                                      Listener Study
                                                   Listeners                         5,000

                                             Average Songs
                                                                                     3,500
                                                Per User


                                                 Percent of
                                                songs never                            65%
                                                 listened to
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


    What makes a good music recommendation?

●   Relevance
●   Novelty / Serendipity
●   Transparency / Trust
●   Reach
●   Context
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


    Music Recommendation is not just
               shopping
●   It is not just for shopping, but...
    ●   Discovery
    ●   Exploration
    ●   Play
    ●   Organization
    ●   Playlisting
    ●   Recommendation for groups
    ●   Devices
●   Doesn't have to look like a spreadsheet!
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


   Context: Tools for exploration
Ishkur's Guide to Electronic Dance Music




 http://techno.org/electronic-music-guide/
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




INTRO CHORUS VERSE BRIDGE OUTRO
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                          Ingestion process
●   Input data source
    ●   Own data, Customer, Labels, UGC, ...
●   Protocol
    ●   Ingestion format
        –    TSV, XML, DDEX, XLS!, …
    ●   Method
        –    FTP, API, ...
    ●   Frequency
        –    Offline processing: Daily / weekly?
        –    Data freshness!
    ●   Documentation
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                        Ingestion process
●   Post-processing
    ●   Data cleaning: Duplicates, normalization
    ●   Allow customer to use its own Ids!
●   Add links to external sources
    ●   Rosetta Stone
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                        Ingestion process
●   Considerations
    ●   Allow customer to use its own IDs when using the
        rec. system.
    ●   How long does it take to process the whole
        collection?
    ●   Incremental updates
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
●   Social-based
●   Content-based
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
       β€œX similar to (or influenced by) Y”
       Editorial metadata (Genre, Decades, Location, …)
       Music Genome
●   Collaborative filtering
●   Social-based
●   Content-based
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
       β€œpeople who listen to X also listen to Y”
●   Social-based
●   Content-based
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
       β€œpeople who listen to X also listen to Y”
●   Social-based
●   Content-based
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
       β€œpeople who listen toRawalso listen to Y”
                             X plays:
●   Social-based
●   Content-based
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
       β€œpeople who listen toRawalso listen to Y”
                             X plays:
●   Social-based                                     Normalize to [5..1]

●   Content-based
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
       β€œpeople who listen toRawalso listen to Y”
                             X plays:
●   Social-based                                     Normalize to [5..1]
                                           Probability distribution:
●   Content-based                          0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
       β€œpeople who listen toRawalso listen to Y”
                             X plays:
●   Social-based                                     Normalize to [5..1]
                                           Probability distribution:
●   Content-based                          0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02
●   Hybrid (combination)                           Binary:
                                                   100100000100000011000001
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
       Matrix Factorization. E.g: SVD, NMF, ...
●   Social-based
●   Content-based
●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


   music recommendation approaches
  ●   Expert-based
  ●   Collaborative filtering
  ●   Social-based
      ●
          WebMIR      [Schedl, 2008]


Content    Reviews           Lyrics          Blogs          Social Tags             Bios           Playlists




  ●   Content-based
  ●   Hybrid (combination)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
●   Social-based
●   Content-based
      β€œX and Y sound similar”
●   Hybrid
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
●   Social-based
●   Content-based
      Audio features
       –    Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon,
            2005], Harmony [Gomez, 2006], ...
      Similarity
       –    KL-divergence: GMM [Aucouturier, 2002]
       –    EMD [Logan, 2001]
       –    Euclidean: PCA [Cano, 2005]
       –    Cosine: mean/var (feature vectors)
       –    Ad-hoc
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
●   Social-based
●   Content-based
      Audio features
       –    Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon,
            2005], Harmony [Gomez, 2006], ...
      Similarity
       –    KL-divergence: GMM [Aucouturier, 2002]
       –    EMD [Logan, 2001]
       –    Euclidean: PCA [Cano, 2005]
                                                                      http://xkcd.com/26/
       –    Cosine: mean/var (feature vectors)
       –    Ad-hoc
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


music recommendation approaches
●   Expert-based
●   Collaborative filtering
●   Social-based
●   Content-based
●   Hybrid
      Weighted (linear combination)
       –     E.g CF * 0.2 + CT * 0.4 + CB * 0.4
      Cascade
       –     E.g 1st apply CF, then reorder by CT or CB
      Switching
      ...
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                        Search
●   Metadata search
        Bruce*


●   Using filters: β€œPopular Irish bands from the 80s”
        popularity:[8.0 TO 10.0] AND
        iso_country:IE AND decade:1980


●   Audio search (and similarity)
    ●   Query by example
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                          Similarity
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                Similarity

  ?                                   ?                                      ?




Using Last.fm-360K dataset
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                               Similarity




Using Last.fm-360K dataset
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


             Similarity' (include feedback)




Using Last.fm-360K dataset
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




   Beyond similarity
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                        Recommendation
●   β€œIf Paul likes Radiohead he might also like X”
    vs.
●   β€œIf Oscar likes Radiohead he might also like Y”
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                        Recommendation
●   β€œIf Paul likes Radiohead he might also like X”
    vs.
●   β€œIf Oscar likes Radiohead he might also like Y”


          SIMILARITY != RECOMMENDATION
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                      Recommendation
●   To whom are we recommending? Phoenix-2 (UK, 2006)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                lamere @ last.fm
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


              mini-lamere's @ last.fm
●
    Clustering (k-means) lamere top-50 artists
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


              mini-lamere's @ last.fm
●
    Clustering (k-means) lamere top-50 artists
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


         @lamere - Radiohead
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


             @lamere - Radiohead




Vs. Radiohead similar artists...
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


             @lamere - Radiohead




Vs. Radiohead similar artists...
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


             @lamere - Radiohead




Vs. Radiohead similar artists...
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


             @lamere - Radiohead




Vs. Radiohead similar artists...
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


      Personalization (Itemization?)
●   ...but also which Radiohead era?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                       Analytics
●   Big data processing
     ●    capture, storage, search, share, analysis and
          visualization
●   (local) Trend detection
●   Tastemakers
●   ...
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




INTRO CHORUS VERSE BRIDGE OUTRO
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Real-world Music Recommendation
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Real-world Music Recommendation
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Contextual Web Crawl
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Audio Processing
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Hybrid Recommendation
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




● 100 million registered users
● 37 million active monthly users

● More than 900,000 songs in catalog

● More than 90,000 artists in catalog

● More than 11 billion thumbs

● More than 1.9 billion stations

● 95% of the collection was played in July 2011
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Curation and Analysis
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


Weighting vectors
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


For unknown artists
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


For popular artists
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




Country:         UK
Record Labels: Acid Jazz, Sony BMG, Columbia
Genres:          Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rock
Years active: 1992 - present
Associated acts: Brand New Heavies, Guru, Julian Perretta
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




Country:         UK
Record Labels: Acid Jazz, Sony BMG, Columbia
Genres:          Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rock
Years active: 1992 - present
Associated acts: Brand New Heavies, Guru, Julian Perretta

Mood:         upbeat, energetic
Rhythm:         120bpm, no rubato, high percusiveness
Key:        Dm
Tags:       acid jazz funk dance
Sounds like: Sereia (Tiefschwarz Radio Edit) by Mundo Azul
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




  β€œI want some upbeat songs from unknown US bands,
                  similar to Radioheadβ€œ


http://ella.bmat.ws/collections/bmat/artists/radiohead/similar/tracks
    ?filter=mood:happy
    +speed:fast
    +iso_country:US
    +popularity:[0.0+TO+4.0]
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




INTRO CHORUS VERSE BRIDGE OUTRO
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                Evaluation
"The key utility measure is user happiness. It seems
  reasonable to assume that relevance of the results is
  the most important factor: blindingly fast, useless
  answers do not make a user happy."

   –    "Introduction to Information Retrieval"
        (Manning, Raghavan, and Schutze, 2008)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




                        RMSE
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




                    RMSE?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




             NO RMSE
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




             NO RMSE
                         (in music)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                     Evaluation
●   Limitations of current metrics (RMSE, P/R, ROC,
    Spearman Rho, Kendall Tau, etc.)
    ●   skewness
        –    performed on test data that users chose to rate
    ●   do not take into account
        –    usefulness
        –    novelty / serendipity
        –    topology of the (item or user) similarity graph
        –    ...
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                   Evaluation
If no RMSE then...?
●   Predictive Accuracy vs. Perceived Quality
●   Does the recommendation help the user? (user
    satisfaction)
    ●   Familiarity vs. Novelty
●   Does the recommendation help the system?
    ●   $$$
    ●   Catalog exposure
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                    NEXT SONG?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                    NEXT SONG?


                                    ?



                                         Mean Reciprocal Rank
                                                  +
                                            User feedback
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




?                                                                                           ?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


          Novelty & Relevance
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


          Novelty & Relevance
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


          Novelty & Relevance
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


          Novelty & Relevance
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


          Novelty & Relevance




                                                                   WTF?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


          Novelty & Relevance




                                                             Emitt Rhodes
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


           Novelty & Relevance


WHY as important as WHAT
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


           Novelty & Relevance


WHY as important as WHAT
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


           Novelty & Relevance


WHY as important as WHAT
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


           Novelty & Relevance


WHY as important as WHAT
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




                             WTF
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


      Other evaluation techniques
                                        rd
How can I evaluate a 3 party recommender:

objective measures:
  coverage, reach




subject measures:
  Focus on precision
  Measure irrelevant results: The WTF test
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


The WTF Test




   Why the Freakomendation?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                      Evaluation
●   Research Datasets
    ●   Million Song Dataset (CB, Social, Lyrics, Tags and
        more)
        http://labrosa.ee.columbia.edu/millionsong/


    ●   Last.fm (CF)
        http://ocelma.net/MusicRecommendationDataset/
         –    Last.fm 360K users <user, artist, total plays>
         –    Last.fm 1K users <user, timestamp, artist, song>
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                    Monitoring
●   Do not monitor (or test) only the Algorithm, but the
    WHOLE recommender system: KPIs
●   Catalog
    ●   % matches against full catalog?
    ●   Ingestion time?
    ●   Availability?
●   Data & Algorithms
    ●   Time computing (e.g. Matrix factorization)?
    ●   Matrix size (e.g. ~10M x ~1M) in memory?
        –    10M vectors with 300 floats per vector β†’ ~11Gb
    ●
        Time computing vector similarity O(n)?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                 Monitoring
USAGE
●   Search
    assert_equal(ID(search('The The')), ID('The The'))


●   Similarity
    assert(similarity(U2, REM) > 0.8)
    assert(similarity(AC/DC, Rebecca Black) < 0.3)


●   Recommendation
    0) create_profile(@ocelma)
    1) assert(similarity(@ocelma, U2) >= 0.8)
    2) dislike(@ocelma, track(U2,Lemon))
    3) assert(similarity(@ocelma, album(U2,Zooropa)) < 0.8)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                    Monitoring
●   (web) API
    ●   Measure query response
        –    Jmeter, Apache Benchmark
    ●   Process real logs
        –    Fake (repeated) queries β†’ fast because using cache?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                    Monitoring
●   (web) API
    ●   Measure query response
        –    Jmeter, Apache Benchmark
    ●   Process real logs
        –    Fake (repeated) queries β†’ fast because using cache?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




INTRO CHORUS VERSE BRIDGE OUTRO
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                                Conclusions
●   Music Recsys is multidisciplinary
    ●   search and filtering, musicology, data mining,
        machine learning, personalization, social networks,
        text processing, complex networks, user interaction,
        information visualization, and signal processing
        (among others!)
●   Music Recsys is important
    ●   These technologies will be integral in helping the next
        generation of music listeners find that next favorite
        song
    ●   Strong industry impact
●   Music Recsys is special
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                          Further research
●   How well do music recommenders work?
    ●   lack of standardized data sets and objective
        evaluation methods

●   How to recognize and incorporate context into
    recommendations?
    ●   listener’s context (exercising, exploring, working,
        driving, relaxing, and so on)
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma


                          Further research
●   How to make recommendations for all music?
    ●   consider all music including new, unknown, and
        unpopular content.




●   What effect will automatic music recommenders
    have on the collective music taste?
recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma




Music Recommendation and Discovery
           Remastered

                                  Tutorial



                        @recsys, 2011

More Related Content

Viewers also liked

Committing to Recommendations Strata 2013
Committing to Recommendations Strata 2013Committing to Recommendations Strata 2013
Committing to Recommendations Strata 2013Eric Colson
Β 
Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...
Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...
Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...Asoka Korale
Β 
ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²
ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²
ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²ΠΈΡ€ΠΈΠ½Π° ΠΎΠ³Π½Π΅Π²Π°
Β 
Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?
Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?
Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?ΠšΠΎΠ½ΡΡ‚Π°Π½Ρ‚ΠΈΠ½ Π—Π°ΠΉΠ½ΡƒΠ»ΠΈΠ½
Β 
Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...
Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...
Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...ΠšΠΎΠ½ΡΡ‚Π°Π½Ρ‚ΠΈΠ½ Π—Π°ΠΉΠ½ΡƒΠ»ΠΈΠ½
Β 
ΠΏΠ°Ρ‚Ρ€Ρ–ΠΎΡ‚ΠΈΡ‡Π½Π΅ виховання
ΠΏΠ°Ρ‚Ρ€Ρ–ΠΎΡ‚ΠΈΡ‡Π½Π΅ вихованняпатріотичнС виховання
ΠΏΠ°Ρ‚Ρ€Ρ–ΠΎΡ‚ΠΈΡ‡Π½Π΅ вихованняRevolschool
Β 
ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³
ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³
ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³Larisalex61
Β 
НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?
НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?
НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?АндрСй Π€Π΅Π΄ΠΎΡ€ΠΎΠ²
Β 
FRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKS
FRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKSFRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKS
FRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKSI3E Technologies
Β 
Building a Recommendation Engine - A Balancing act
Building a Recommendation Engine - A Balancing actBuilding a Recommendation Engine - A Balancing act
Building a Recommendation Engine - A Balancing actElad Rosenheim
Β 
ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...
ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...
ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...stecenko_nm
Β 
Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014rpbrehm
Β 
Solving the AL Chicken-and-Egg Corpus and Model Problem
Solving the AL Chicken-and-Egg Corpus and Model ProblemSolving the AL Chicken-and-Egg Corpus and Model Problem
Solving the AL Chicken-and-Egg Corpus and Model ProblemDain Kaplan
Β 
Π‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ
Π‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈΠ‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ
Π‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈΠ•Π»Π΅Π½Π° Π‘ΠΌΡƒΡ‚Π½Π΅Π²Π°
Β 
Challenge@rule ml2015 rule based recommender systems for the Web of Data
Challenge@rule ml2015 rule based recommender systems for the Web of DataChallenge@rule ml2015 rule based recommender systems for the Web of Data
Challenge@rule ml2015 rule based recommender systems for the Web of DataRuleML
Β 
Multi Criteria Recommender Systems - Overview
Multi Criteria Recommender Systems - OverviewMulti Criteria Recommender Systems - Overview
Multi Criteria Recommender Systems - OverviewDavide Giannico
Β 
Your own recommendation engine with neo4j and reco4php - DPC16
Your own recommendation engine with neo4j and reco4php - DPC16Your own recommendation engine with neo4j and reco4php - DPC16
Your own recommendation engine with neo4j and reco4php - DPC16Christophe Willemsen
Β 
Summary of a Recommender Systems Survey paper
Summary of a Recommender Systems Survey paperSummary of a Recommender Systems Survey paper
Summary of a Recommender Systems Survey paperChangsung Moon
Β 
Profile injection attack detection in recommender system
Profile injection attack detection in recommender systemProfile injection attack detection in recommender system
Profile injection attack detection in recommender systemASHISH PANNU
Β 

Viewers also liked (20)

Committing to Recommendations Strata 2013
Committing to Recommendations Strata 2013Committing to Recommendations Strata 2013
Committing to Recommendations Strata 2013
Β 
Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...
Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...
Recommender Algorithm for PRBT BiPartite Networks - IESL 18 Oct 2016_final_us...
Β 
ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²
ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²
ΠΎΠ³Π½Π΅Π²Π° ΠΈ. врСмя Π½ΠΎΠ²Ρ‹Ρ… ΠΌΠ°Ρ€ΡˆΡ€ΡƒΡ‚ΠΎΠ²
Β 
Muzei mira
Muzei miraMuzei mira
Muzei mira
Β 
Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?
Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?
Π’Π΅Π°Ρ‚Ρ€ 2.0. Π“Π΄Π΅ ΠΈ ΠΊΠ°ΠΊ Π½Π°ΠΌ Ρ€Π°Π±ΠΎΡ‚Π°Ρ‚ΡŒ Π² Π˜Π½Ρ‚Π΅Ρ€Π½Π΅Ρ‚Π΅?
Β 
Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...
Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...
Π£Ρ‡Ρ€Π΅ΠΆΠ΄Π΅Π½ΠΈΠ΅ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹: финансовая ΡƒΡΡ‚ΠΎΠΉΡ‡ΠΈΠ²ΠΎΡΡ‚ΡŒ, рСсурсноС обСспСчСниС, эффСктив...
Β 
ΠΏΠ°Ρ‚Ρ€Ρ–ΠΎΡ‚ΠΈΡ‡Π½Π΅ виховання
ΠΏΠ°Ρ‚Ρ€Ρ–ΠΎΡ‚ΠΈΡ‡Π½Π΅ вихованняпатріотичнС виховання
ΠΏΠ°Ρ‚Ρ€Ρ–ΠΎΡ‚ΠΈΡ‡Π½Π΅ виховання
Β 
ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³
ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³
ΠΊΠ°ΠΊ ΡΠΎΠ·Π΄Π°Ρ‚ΡŒ свой Π±Π»ΠΎΠ³
Β 
НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?
НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?
НуТны Π»ΠΈ Π±Π»ΠΎΠ³ΠΈ Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊΠ°ΠΌ?
Β 
FRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKS
FRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKSFRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKS
FRIENDBOOK: A SEMANTIC-BASED FRIEND RECOMMENDATION SYSTEM FOR SOCIAL NETWORKS
Β 
Building a Recommendation Engine - A Balancing act
Building a Recommendation Engine - A Balancing actBuilding a Recommendation Engine - A Balancing act
Building a Recommendation Engine - A Balancing act
Β 
ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...
ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...
ΠœΠΎΠ΄ΡƒΠ»ΡŒ1. Π’Π΅ΠΌΠ° 3. ΠŸΡ€ΠΎΠ±Π»Π΅ΠΌΠ° ΠΌΠ΅Ρ‚ΠΈ Π² ΠΏΠ΅Π΄Π°Π³ΠΎΠ³Ρ–Ρ†Ρ–. ΠœΠ΅Ρ‚Π° виховання Π² Π½Π°Ρ†Ρ–ΠΎΠ½Π°Π»ΡŒΠ½Ρ–ΠΉ ш...
Β 
Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014Recommender.system.presentation.pjug.01.21.2014
Recommender.system.presentation.pjug.01.21.2014
Β 
Solving the AL Chicken-and-Egg Corpus and Model Problem
Solving the AL Chicken-and-Egg Corpus and Model ProblemSolving the AL Chicken-and-Egg Corpus and Model Problem
Solving the AL Chicken-and-Egg Corpus and Model Problem
Β 
Π‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ
Π‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈΠ‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ
Π‘ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅Ρ‡Π½ΠΎΠ΅ пространство ΠΎΡ‚ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ†ΠΈΠΈ ΠΊ Ρ€Π΅Π°Π»ΠΈΠ·Π°Ρ†ΠΈΠΈ
Β 
Challenge@rule ml2015 rule based recommender systems for the Web of Data
Challenge@rule ml2015 rule based recommender systems for the Web of DataChallenge@rule ml2015 rule based recommender systems for the Web of Data
Challenge@rule ml2015 rule based recommender systems for the Web of Data
Β 
Multi Criteria Recommender Systems - Overview
Multi Criteria Recommender Systems - OverviewMulti Criteria Recommender Systems - Overview
Multi Criteria Recommender Systems - Overview
Β 
Your own recommendation engine with neo4j and reco4php - DPC16
Your own recommendation engine with neo4j and reco4php - DPC16Your own recommendation engine with neo4j and reco4php - DPC16
Your own recommendation engine with neo4j and reco4php - DPC16
Β 
Summary of a Recommender Systems Survey paper
Summary of a Recommender Systems Survey paperSummary of a Recommender Systems Survey paper
Summary of a Recommender Systems Survey paper
Β 
Profile injection attack detection in recommender system
Profile injection attack detection in recommender systemProfile injection attack detection in recommender system
Profile injection attack detection in recommender system
Β 

More from Paul Lamere

Sxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to musicSxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to musicPaul Lamere
Β 
Beyond the Play Button - The future of listening
Beyond the Play Button - The future of listeningBeyond the Play Button - The future of listening
Beyond the Play Button - The future of listeningPaul Lamere
Β 
Echo nest-api-boston-2012
Echo nest-api-boston-2012Echo nest-api-boston-2012
Echo nest-api-boston-2012Paul Lamere
Β 
I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what? I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what? Paul Lamere
Β 
Data Mining Music
Data Mining MusicData Mining Music
Data Mining MusicPaul Lamere
Β 
The Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack DayThe Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack DayPaul Lamere
Β 
Using Visualizations for Music Discovery
Using Visualizations for Music DiscoveryUsing Visualizations for Music Discovery
Using Visualizations for Music DiscoveryPaul Lamere
Β 
Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.Paul Lamere
Β 
Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)Paul Lamere
Β 
Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)Paul Lamere
Β 

More from Paul Lamere (10)

Sxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to musicSxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to music
Β 
Beyond the Play Button - The future of listening
Beyond the Play Button - The future of listeningBeyond the Play Button - The future of listening
Beyond the Play Button - The future of listening
Β 
Echo nest-api-boston-2012
Echo nest-api-boston-2012Echo nest-api-boston-2012
Echo nest-api-boston-2012
Β 
I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what? I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what?
Β 
Data Mining Music
Data Mining MusicData Mining Music
Data Mining Music
Β 
The Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack DayThe Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack Day
Β 
Using Visualizations for Music Discovery
Using Visualizations for Music DiscoveryUsing Visualizations for Music Discovery
Using Visualizations for Music Discovery
Β 
Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.
Β 
Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)
Β 
Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)
Β 

Recently uploaded

My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
Β 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentationphoebematthew05
Β 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
Β 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
Β 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
Β 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
Β 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
Β 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
Β 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
Β 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
Β 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
Β 
Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)
Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)
Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)Wonjun Hwang
Β 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
Β 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
Β 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
Β 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
Β 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
Β 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
Β 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
Β 

Recently uploaded (20)

My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Β 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentation
Β 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
Β 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Β 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Β 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
Β 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Β 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
Β 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
Β 
Hot Sexy call girls in Panjabi Bagh πŸ” 9953056974 πŸ” Delhi escort Service
Hot Sexy call girls in Panjabi Bagh πŸ” 9953056974 πŸ” Delhi escort ServiceHot Sexy call girls in Panjabi Bagh πŸ” 9953056974 πŸ” Delhi escort Service
Hot Sexy call girls in Panjabi Bagh πŸ” 9953056974 πŸ” Delhi escort Service
Β 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
Β 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
Β 
Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)
Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)
Bun (KitWorks Team Study λ…Έλ³„λ§ˆλ£¨ λ°œν‘œ 2024.4.22)
Β 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
Β 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
Β 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
Β 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
Β 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
Β 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
Β 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
Β 

Music Recommendation and Discovery

  • 1. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music Recommendation and Discovery Remastered Tutorial @recsys, 2011
  • 2. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @plamere @ocelma
  • 3. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma INTRO CHORUS VERSE BRIDGE OUTRO
  • 4. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma INTRO CHORUS VERSE BRIDGE OUTRO
  • 5. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music Recommendation is important How many songs fit in my pocket? 10 Songs 1,000 Songs 10,000,000 Songs 1979 2001 2011
  • 6. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What's so special about music? ● Huge item space ● Very low cost per item ● Many item types ● Low consumption time ● Very high per-item reuse ● Highly passionate users ● Highly contextual usage ● Consumed in sequences ● Large personal collections ● Doesn't require our full attention ● Highly Social
  • 7. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music recommendation is broken ... If you like Britney Spears you might like... ...Report on Pre-War Intelligence Let's look at some of the issues ....
  • 8. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation? ● Relevance ● Novelty / Serendipity ● Transparency / Trust ● Reach ● Context
  • 9. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation? ● Relevance ● Novelty / Serendipity ● Transparency / Trust ● Reach ● Context
  • 10. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance
  • 11. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance – cold start new or unpopular items If you like Gregorian Chants you might like Green Day
  • 12. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Cold Start – New User - Enrollment
  • 13. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma New User – Implicit taste data The Audioscrobbler
  • 14. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance – Metadata Mismatches
  • 15. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance – Metadata Mismatches Why?
  • 16. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance - The grey sheep problem
  • 17. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Relevance – Cultural Mismatches
  • 18. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation? ● Relevance ● Novelty / Serendipity ● Transparency / Trust ● Reach ● Context
  • 19. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty and Serendipity
  • 20. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Popularity Bias - The Harry Potter Effect
  • 21. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma ...also known as the Coldplay effect
  • 22. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty / Serendipity – the enemy High stakes competitions focused on relevance can reduce novelty and serendipity
  • 23. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation? ● Relevance ● Novelty / Serendipity ● Transparency / Trust ● Reach ● Context
  • 24. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma The Opacity Problem β€œIf you like NiN you might like Johnny Cash” Why???
  • 25. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Is this a good recommendation? If you like Norah Jones ... You might like Ravi Shankar
  • 26. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Is this a good recommendation? If you like Norah Jones ... You might like her father, Ravi Shankar
  • 27. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Brutal Death Metal Quiz ??????? Photo cc by Mithrandir3
  • 28. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Brutal Death Metal Quiz
  • 29. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Hacking the recommender
  • 30. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation? ● Relevance ● Novelty / Serendipity ● Transparency / Trust ● Reach ● Context
  • 31. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Help! I’m stuck in the head The limited reach of music recommendation 48% of recommendations Popularity 0% of recommendations 52% of recommendations 83 Artists 6,659 Artists 239,798 Artists Study by Dr. Oscar Celma - MTG UPF Sales Rank
  • 32. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music Discovery Challenge Personal discovery a challenge too Listener Study Listeners 5,000 Average Songs 3,500 Per User Percent of songs never 65% listened to
  • 33. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma What makes a good music recommendation? ● Relevance ● Novelty / Serendipity ● Transparency / Trust ● Reach ● Context
  • 34. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music Recommendation is not just shopping ● It is not just for shopping, but... ● Discovery ● Exploration ● Play ● Organization ● Playlisting ● Recommendation for groups ● Devices ● Doesn't have to look like a spreadsheet!
  • 35. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Context: Tools for exploration Ishkur's Guide to Electronic Dance Music http://techno.org/electronic-music-guide/
  • 36. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma INTRO CHORUS VERSE BRIDGE OUTRO
  • 37. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 38. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 39. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process ● Input data source ● Own data, Customer, Labels, UGC, ... ● Protocol ● Ingestion format – TSV, XML, DDEX, XLS!, … ● Method – FTP, API, ... ● Frequency – Offline processing: Daily / weekly? – Data freshness! ● Documentation
  • 40. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process ● Post-processing ● Data cleaning: Duplicates, normalization ● Allow customer to use its own Ids! ● Add links to external sources ● Rosetta Stone
  • 41. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Ingestion process ● Considerations ● Allow customer to use its own IDs when using the rec. system. ● How long does it take to process the whole collection? ● Incremental updates
  • 42. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 43. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 44. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● Content-based ● Hybrid (combination)
  • 45. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based β€œX similar to (or influenced by) Y” Editorial metadata (Genre, Decades, Location, …) Music Genome ● Collaborative filtering ● Social-based ● Content-based ● Hybrid (combination)
  • 46. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering β€œpeople who listen to X also listen to Y” ● Social-based ● Content-based ● Hybrid (combination)
  • 47. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering β€œpeople who listen to X also listen to Y” ● Social-based ● Content-based ● Hybrid (combination)
  • 48. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering β€œpeople who listen toRawalso listen to Y” X plays: ● Social-based ● Content-based ● Hybrid (combination)
  • 49. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering β€œpeople who listen toRawalso listen to Y” X plays: ● Social-based Normalize to [5..1] ● Content-based ● Hybrid (combination)
  • 50. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering β€œpeople who listen toRawalso listen to Y” X plays: ● Social-based Normalize to [5..1] Probability distribution: ● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02 ● Hybrid (combination)
  • 51. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering β€œpeople who listen toRawalso listen to Y” X plays: ● Social-based Normalize to [5..1] Probability distribution: ● Content-based 0.32 , 0.24 , 0.18 , 0.1 , 0.07 , 0.04, 0.03 , 0.02 ● Hybrid (combination) Binary: 100100000100000011000001
  • 52. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering Matrix Factorization. E.g: SVD, NMF, ... ● Social-based ● Content-based ● Hybrid (combination)
  • 53. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● WebMIR [Schedl, 2008] Content Reviews Lyrics Blogs Social Tags Bios Playlists ● Content-based ● Hybrid (combination)
  • 54. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● Content-based β€œX and Y sound similar” ● Hybrid
  • 55. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● Content-based Audio features – Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon, 2005], Harmony [Gomez, 2006], ... Similarity – KL-divergence: GMM [Aucouturier, 2002] – EMD [Logan, 2001] – Euclidean: PCA [Cano, 2005] – Cosine: mean/var (feature vectors) – Ad-hoc
  • 56. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● Content-based Audio features – Bag-of-frames (MFCC) [Aucouturier, 2004], Rhythm [Gouyon, 2005], Harmony [Gomez, 2006], ... Similarity – KL-divergence: GMM [Aucouturier, 2002] – EMD [Logan, 2001] – Euclidean: PCA [Cano, 2005] http://xkcd.com/26/ – Cosine: mean/var (feature vectors) – Ad-hoc
  • 57. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma music recommendation approaches ● Expert-based ● Collaborative filtering ● Social-based ● Content-based ● Hybrid Weighted (linear combination) – E.g CF * 0.2 + CT * 0.4 + CB * 0.4 Cascade – E.g 1st apply CF, then reorder by CT or CB Switching ...
  • 58. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 59. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 60. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Search ● Metadata search Bruce* ● Using filters: β€œPopular Irish bands from the 80s” popularity:[8.0 TO 10.0] AND iso_country:IE AND decade:1980 ● Audio search (and similarity) ● Query by example
  • 61. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity
  • 62. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity ? ? ? Using Last.fm-360K dataset
  • 63. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity Using Last.fm-360K dataset
  • 64. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Similarity' (include feedback) Using Last.fm-360K dataset
  • 65. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Beyond similarity
  • 66. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation ● β€œIf Paul likes Radiohead he might also like X” vs. ● β€œIf Oscar likes Radiohead he might also like Y”
  • 67. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation ● β€œIf Paul likes Radiohead he might also like X” vs. ● β€œIf Oscar likes Radiohead he might also like Y” SIMILARITY != RECOMMENDATION
  • 68. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Recommendation ● To whom are we recommending? Phoenix-2 (UK, 2006)
  • 69. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma lamere @ last.fm
  • 70. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma mini-lamere's @ last.fm ● Clustering (k-means) lamere top-50 artists
  • 71. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma mini-lamere's @ last.fm ● Clustering (k-means) lamere top-50 artists
  • 72. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - Radiohead
  • 73. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - Radiohead Vs. Radiohead similar artists...
  • 74. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - Radiohead Vs. Radiohead similar artists...
  • 75. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - Radiohead Vs. Radiohead similar artists...
  • 76. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma @lamere - Radiohead Vs. Radiohead similar artists...
  • 77. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Personalization (Itemization?) ● ...but also which Radiohead era?
  • 78. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Analytics ● Big data processing ● capture, storage, search, share, analysis and visualization ● (local) Trend detection ● Tastemakers ● ...
  • 79. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma INTRO CHORUS VERSE BRIDGE OUTRO
  • 80. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Real-world Music Recommendation
  • 81. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Real-world Music Recommendation
  • 82. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Contextual Web Crawl
  • 83. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Audio Processing
  • 84. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Hybrid Recommendation
  • 85. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma ● 100 million registered users ● 37 million active monthly users ● More than 900,000 songs in catalog ● More than 90,000 artists in catalog ● More than 11 billion thumbs ● More than 1.9 billion stations ● 95% of the collection was played in July 2011
  • 86. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Curation and Analysis
  • 87. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Weighting vectors
  • 88. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma For unknown artists
  • 89. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma For popular artists
  • 90. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 91. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 92. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Country: UK Record Labels: Acid Jazz, Sony BMG, Columbia Genres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rock Years active: 1992 - present Associated acts: Brand New Heavies, Guru, Julian Perretta
  • 93. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Country: UK Record Labels: Acid Jazz, Sony BMG, Columbia Genres: Funk, Disco, Acid Jazz, Jazz Fusion, Pop-rock Years active: 1992 - present Associated acts: Brand New Heavies, Guru, Julian Perretta Mood: upbeat, energetic Rhythm: 120bpm, no rubato, high percusiveness Key: Dm Tags: acid jazz funk dance Sounds like: Sereia (Tiefschwarz Radio Edit) by Mundo Azul
  • 94. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma β€œI want some upbeat songs from unknown US bands, similar to Radioheadβ€œ http://ella.bmat.ws/collections/bmat/artists/radiohead/similar/tracks ?filter=mood:happy +speed:fast +iso_country:US +popularity:[0.0+TO+4.0]
  • 95. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma INTRO CHORUS VERSE BRIDGE OUTRO
  • 96. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation "The key utility measure is user happiness. It seems reasonable to assume that relevance of the results is the most important factor: blindingly fast, useless answers do not make a user happy." – "Introduction to Information Retrieval" (Manning, Raghavan, and Schutze, 2008)
  • 97. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma RMSE
  • 98. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma RMSE?
  • 99. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NO RMSE
  • 100. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NO RMSE (in music)
  • 101. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation ● Limitations of current metrics (RMSE, P/R, ROC, Spearman Rho, Kendall Tau, etc.) ● skewness – performed on test data that users chose to rate ● do not take into account – usefulness – novelty / serendipity – topology of the (item or user) similarity graph – ...
  • 102. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation If no RMSE then...? ● Predictive Accuracy vs. Perceived Quality ● Does the recommendation help the user? (user satisfaction) ● Familiarity vs. Novelty ● Does the recommendation help the system? ● $$$ ● Catalog exposure
  • 103. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NEXT SONG?
  • 104. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma NEXT SONG? ? Mean Reciprocal Rank + User feedback
  • 105. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma ? ?
  • 106. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 107. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  • 108. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  • 109. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  • 110. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance
  • 111. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance WTF?
  • 112. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance Emitt Rhodes
  • 113. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance WHY as important as WHAT
  • 114. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance WHY as important as WHAT
  • 115. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance WHY as important as WHAT
  • 116. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Novelty & Relevance WHY as important as WHAT
  • 117. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma WTF
  • 118. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Other evaluation techniques rd How can I evaluate a 3 party recommender: objective measures: coverage, reach subject measures: Focus on precision Measure irrelevant results: The WTF test
  • 119. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma The WTF Test Why the Freakomendation?
  • 120. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Evaluation ● Research Datasets ● Million Song Dataset (CB, Social, Lyrics, Tags and more) http://labrosa.ee.columbia.edu/millionsong/ ● Last.fm (CF) http://ocelma.net/MusicRecommendationDataset/ – Last.fm 360K users <user, artist, total plays> – Last.fm 1K users <user, timestamp, artist, song>
  • 121. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 122. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma
  • 123. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring ● Do not monitor (or test) only the Algorithm, but the WHOLE recommender system: KPIs ● Catalog ● % matches against full catalog? ● Ingestion time? ● Availability? ● Data & Algorithms ● Time computing (e.g. Matrix factorization)? ● Matrix size (e.g. ~10M x ~1M) in memory? – 10M vectors with 300 floats per vector β†’ ~11Gb ● Time computing vector similarity O(n)?
  • 124. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring USAGE ● Search assert_equal(ID(search('The The')), ID('The The')) ● Similarity assert(similarity(U2, REM) > 0.8) assert(similarity(AC/DC, Rebecca Black) < 0.3) ● Recommendation 0) create_profile(@ocelma) 1) assert(similarity(@ocelma, U2) >= 0.8) 2) dislike(@ocelma, track(U2,Lemon)) 3) assert(similarity(@ocelma, album(U2,Zooropa)) < 0.8)
  • 125. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring ● (web) API ● Measure query response – Jmeter, Apache Benchmark ● Process real logs – Fake (repeated) queries β†’ fast because using cache?
  • 126. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Monitoring ● (web) API ● Measure query response – Jmeter, Apache Benchmark ● Process real logs – Fake (repeated) queries β†’ fast because using cache?
  • 127. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma INTRO CHORUS VERSE BRIDGE OUTRO
  • 128. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Conclusions ● Music Recsys is multidisciplinary ● search and filtering, musicology, data mining, machine learning, personalization, social networks, text processing, complex networks, user interaction, information visualization, and signal processing (among others!) ● Music Recsys is important ● These technologies will be integral in helping the next generation of music listeners find that next favorite song ● Strong industry impact ● Music Recsys is special
  • 129. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Further research ● How well do music recommenders work? ● lack of standardized data sets and objective evaluation methods ● How to recognize and incorporate context into recommendations? ● listener’s context (exercising, exploring, working, driving, relaxing, and so on)
  • 130. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Further research ● How to make recommendations for all music? ● consider all music including new, unknown, and unpopular content. ● What effect will automatic music recommenders have on the collective music taste?
  • 131. recsys 2011 | music recommendation and discovery tutorial | paul lamere & oscar celma Music Recommendation and Discovery Remastered Tutorial @recsys, 2011