When I first decided to memorize Psalm 119, I thought it would be a good idea to have some good Puritan and Reformed books to read alongside so as to help me understand and apply it all in a better way. In John Calvin's Sermons on Psalm 119, he exposits every verse very profitably. I came to what he has to say about this well-known verse, and thought it was worth sharing.
Verse 11: I have stored up your word in my heart that I might not sin against you.
When David speaks after this manner, “I have hid your word or promise in my heart,” he well declares, that if we have but only a wandering knowledge, that it will not hold us in, but that the Devil will by and by win upon us to oppress us, with temptations, and in the end to cast us down headlong. What must we then do? It is not enough that we have been at church, and heard what hath been there said to us, and that every one of us has mumbled to himself some one thing or another, but the word of God must be settled in us and be hid in our heart, to wit, that it may there be residing and continually abiding: and to have received it with such an affection, as that it be as it were imprinted in us. If this be not so, sin will reign in us, for it hath natural habitation with us: For all our senses are wicked and corrupt, all our wills and desires are enemies to God, unless God’s word be well hidden in our hearts.
Moreover, we are to understand, that David did not here vaunt himself of his own power and strength, as though he were in admiration thereof: but the spirit of God speaking by his mouth, intends to give us a mirror, wherein we must be confirmed, to wit, that we must not have only our ears beaten with the Doctrine of salvation, and receive it in our brain: but that it should be hidden in our heart, to wit, that we should lay it up as in a Treasure house.
For this saying, to hide, signifies that David did not study to be ambitious to set forth himself, and to make a glorious show before men: but that he had God for a witness of that secret desire which was before him. He never looked to worldly creatures, but being content that he had so great a Treasure, he knew full well that God who had given it him, would so surely and safely guard it, as that it should not be laid open to Satan to be taken away.
Saint Paul, in 1 Timothy 1:19, also declares to us, that the chest wherein this treasure must be hid, is a good conscience. For it is said, that many being void of this good conscience, have lost also their faith, and have been robbed thereof. As if a man should forsake his goods and put them in an open place, without shutting of any door, it were an easy matter for thieves to come in and to steal and deprive and spoil him of all: Even so, if we leave at random to Satan the Treasure which GOD hath given unto us in his word, if it is not hidden in this good conscience, and in the very bottom of our heart as David here speaks, we shall be spoiled thereof.